一：场景复现：

1.1：关键信息：

执行K8S相关命令显示关键信息

Name: "calico-config", Namespace: "kube-system"
from server for: "calico.yaml": configmaps "calico-config" is forbidden: User "system:node:vboxnode3ccccccttttttchenyang" cannot ge API group "" in the namespace "kube-system": no relationship found between node 'vboxnode3ccccccttttttchenyang' and this object
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "bgpconfigurations.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "bgpconfigurations.crd.projectcalico.org" is forbiddnode3ccccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "bgppeers.crd.projectcalico.org", Namespace: ""

1.2：全异常日志输出：

[root@vboxnode3ccccccttttttchenyang ~]# kubectl apply -f calico.yaml
^[[Aerror parsing calico.yaml: error converting YAML to JSON: yaml: line 153: found character that cannot start any token
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "/v1, Resource=configmaps", GroupVersionKind: "/v1, Kind=ConfigMap"
Name: "calico-config", Namespace: "kube-system"
from server for: "calico.yaml": configmaps "calico-config" is forbidden: User "system:node:vboxnode3ccccccttttttchenyang" cannot ge API group "" in the namespace "kube-system": no relationship found between node 'vboxnode3ccccccttttttchenyang' and this object
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "bgpconfigurations.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "bgpconfigurations.crd.projectcalico.org" is forbiddnode3ccccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "bgppeers.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "bgppeers.crd.projectcalico.org" is forbidden: User ccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "blockaffinities.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "blockaffinities.crd.projectcalico.org" is forbiddende3ccccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "caliconodestatuses.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "caliconodestatuses.crd.projectcalico.org" is forbidxnode3ccccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scop
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "clusterinformations.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "clusterinformations.crd.projectcalico.org" is forbioxnode3ccccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster sco
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "felixconfigurations.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "felixconfigurations.crd.projectcalico.org" is forbioxnode3ccccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster sco
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "globalnetworkpolicies.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "globalnetworkpolicies.crd.projectcalico.org" is forvboxnode3ccccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster s
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "globalnetworksets.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "globalnetworksets.crd.projectcalico.org" is forbiddnode3ccccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "hostendpoints.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "hostendpoints.crd.projectcalico.org" is forbidden: 3ccccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "ipamblocks.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "ipamblocks.crd.projectcalico.org" is forbidden: Useccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "ipamconfigs.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "ipamconfigs.crd.projectcalico.org" is forbidden: Uscccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "ipamhandles.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "ipamhandles.crd.projectcalico.org" is forbidden: Uscccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "ippools.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "ippools.crd.projectcalico.org" is forbidden: User "cttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "ipreservations.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "ipreservations.crd.projectcalico.org" is forbidden:e3ccccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "kubecontrollersconfigurations.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "kubecontrollersconfigurations.crd.projectcalico.orgem:node:vboxnode3ccccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the c
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "networkpolicies.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "networkpolicies.crd.projectcalico.org" is forbiddende3ccccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "apiextensions.k8s.io/v1, Resource=customresourcedefinitions", GroupVersionKind: "apiextensions.k8s.io/v1, Kind=CustomRes
Name: "networksets.crd.projectcalico.org", Namespace: ""
from server for: "calico.yaml": customresourcedefinitions.apiextensions.k8s.io "networksets.crd.projectcalico.org" is forbidden: Uscccccttttttchenyang" cannot get resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterroles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRole"
Name: "calico-kube-controllers", Namespace: ""
from server for: "calico.yaml": clusterroles.rbac.authorization.k8s.io "calico-kube-controllers" is forbidden: User "system:node:vbg" cannot get resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterrolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=Clust
Name: "calico-kube-controllers", Namespace: ""
from server for: "calico.yaml": clusterrolebindings.rbac.authorization.k8s.io "calico-kube-controllers" is forbidden: User "system:chenyang" cannot get resource "clusterrolebindings" in API group "rbac.authorization.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterroles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRole"
Name: "calico-node", Namespace: ""
from server for: "calico.yaml": clusterroles.rbac.authorization.k8s.io "calico-node" is forbidden: User "system:node:vboxnode3ccccct resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterrolebindings", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=Clust
Name: "calico-node", Namespace: ""
from server for: "calico.yaml": clusterrolebindings.rbac.authorization.k8s.io "calico-node" is forbidden: User "system:node:vboxnodnnot get resource "clusterrolebindings" in API group "rbac.authorization.k8s.io" at the cluster scope

二：解决流程：

2.1：更新 kubeconfig：

2.1.1：执行命令：

export KUBECONFIG=/root/.kube/config
chmod g-r /root/.kube/config

2.2：再次执行之前的异常命令：

[root@vboxnode3ccccccttttttchenyang ~]#  kubectl get deployment
Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
[root@vboxnode3ccccccttttttchenyang ~]# kubectl apply -f calico.yaml
Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")

2.3：k8s配置操作config相关命令：

这一步应该在安装K8S kubeadm init后面执行，之前安装忘记这一步。

 mkdir -p $HOME/.kube
 sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
 sudo chown $(id -u):$(id -g) $HOME/.kube/config

[root@vboxnode3ccccccttttttchenyang ~]# mkdir -p $HOME/.kube
[root@vboxnode3ccccccttttttchenyang ~]#
[root@vboxnode3ccccccttttttchenyang ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
cp：是否覆盖"/root/.kube/config"？ y
[root@vboxnode3ccccccttttttchenyang ~]# sudo chown $(id-u):$(id -g) $HOME/.kube/config
在这里插入图片描述

2.4：重新执行之前操作失效命令：

可以看见都已经生效：

kubectl get nodes -o wide
kubectl apply -f calico.yaml

[root@vboxnode3ccccccttttttchenyang ~]# kubectl get nodes -o wide
NAME                            STATUS     ROLES                  AGE   VERSION   INTERNAL-IP      EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION                CONTAINER-RUNTIME
chenyang-mine-vbox02            NotReady   <none>                 19h   v1.21.0   192.168.56.102   <none>        CentOS Linux 7 (Core)   3.10.0-1160.88.1.el7.x86_64   docker://23.0.5
vboxnode3ccccccttttttchenyang   NotReady   control-plane,master   19h   v1.21.0   192.168.56.103   <none>        CentOS Linux 7 (Core)   3.10.0-1160.90.1.el7.x86_64   docker://23.0.5
[root@vboxnode3ccccccttttttchenyang ~]# kubectl apply -f calico.yaml
configmap/calico-config created
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created
clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created
clusterrole.rbac.authorization.k8s.io/calico-node created
clusterrolebinding.rbac.authorization.k8s.io/calico-node created
error: error parsing calico.yaml: error converting YAML to JSON: yaml: line 153: found character that cannot start any token
[root@vboxnode3ccccccttttttchenyang ~]#

2.5：[解决]：重新启动机器后发现kubectl apply -f相关命令又失效无权限相关：

2.5.1：异常日志显示：

[root@vboxnode3ccccccttttttchenyang ~]# vim calico.yaml
-bash: vim: 未找到命令
[root@vboxnode3ccccccttttttchenyang ~]# vi calico.yaml
[root@vboxnode3ccccccttttttchenyang ~]# kubectl apply -f calico.yaml
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "policy/v1, Resource=poddisruptionbudgets", GroupVersionKind: "policy/v1, Kind=PodDisruptionBudget"
Name: "calico-kube-controllers", Namespace: "kube-system"
from server for: "calico.yaml": poddisruptionbudgets.policy "calico-kube-controllers" is forbidden: User "system:node:vboxnode3cccc                                                                 ccttttttchenyang" cannot get resource "poddisruptionbudgets" in API group "policy" in the namespace "kube-system"
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "/v1, Resource=serviceaccounts", GroupVersionKind: "/v1, Kind=ServiceAccount"
Name: "calico-kube-controllers", Namespace: "kube-system"
from server for: "calico.yaml": serviceaccounts "calico-kube-controllers" is forbidden: User "system:node:vboxnode3ccccccttttttchen                                                                 yang" cannot get resource "serviceaccounts" in API group "" in the namespace "kube-system": can only create tokens for individual s                                                                 ervice accounts
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "/v1, Resource=serviceaccounts", GroupVersionKind: "/v1, Kind=ServiceAccount"
Name: "calico-node", Namespace: "kube-system"
from server for: "calico.yaml": serviceaccounts "calico-node" is forbidden: User "system:node:vboxnode3ccccccttttttchenyang" cannot                                                                  get resource "serviceaccounts" in API group "" in the namespace "kube-system": can only create tokens for individual service accou                                                                 nts

2.5.2：解决方式：执行export KUBECONFIG=/root/.kube/config / chmod g-r /root/.kube/config

export KUBECONFIG=/root/.kube/config
chmod g-r /root/.kube/config
kubectl apply -f calico.yaml

[root@vboxnode3ccccccttttttchenyang ~]# export KUBECONFIG=/root/.kube/config
[root@vboxnode3ccccccttttttchenyang ~]# chmod g-r /root/.kube/config
[root@vboxnode3ccccccttttttchenyang ~]# kubectl apply -f calico.yaml

kubectl delete pod coredns-545d6fc579-6ddhj coredns-545d6fc579-nkk68 -n kube-system

[root@vboxnode3ccccccttttttchenyang ~]# export KUBECONFIG=/root/.kube/config
[root@vboxnode3ccccccttttttchenyang ~]# chmod g-r /root/.kube/config
[root@vboxnode3ccccccttttttchenyang ~]# kubectl apply -f calico.yaml
poddisruptionbudget.policy/calico-kube-controllers created
serviceaccount/calico-kube-controllers created
serviceaccount/calico-node created
configmap/calico-config unchanged
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org configured
customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org configured
clusterrole.rbac.authorization.k8s.io/calico-kube-controllers configured
clusterrole.rbac.authorization.k8s.io/calico-node configured
clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers unchanged
clusterrolebinding.rbac.authorization.k8s.io/calico-node unchanged
daemonset.apps/calico-node created
deployment.apps/calico-kube-controllers created
[root@vboxnode3ccccccttttttchenyang ~]#