CI/CD—K8S 基本理解与部署

1 K8S 是什么

Kubernetes 是一款容器的编排调度工具，来源于 Google 开源的 Brog 系统。Kubernetes简称K8S，是用8代替8个字符 “ubernete” 而成的缩写，用于管理云平台中多个主机上的容器化的应用，Kubernetes 的目标是让部署容器化的应用简单并且高效，Kubernetes提供了应用部署，规划，更新，维护的一种机制。

在这里插入图片描述

集群中的各种角色
在这里插入图片描述
apiserver 所有服务访问的唯一入口，提供认证、授权、访问控制、API 注册和发现等机制
controller manager 负责维护集群的状态，比如副本期望数量、故障检测、自动扩展、滚动更新等
scheduler 负责资源的调度，按照预定的调度策略将 Pod 调度到相应的机器上
etcd 键值对数据库，保存了整个集群的状态
kubelet 负责维护容器的生命周期，同时也负责 Volume 和网络的管理
kube-proxy 负责为 Service 提供 cluster 内部的服务发现和负载均衡
Container runtime 负责镜像管理以及 Pod 和容器的真正运行

nginx 要调度到哪个工作节点呢？由scheduler监听API server发现有新的创建应用，通过API server 到 etcd 拿数据，然后 controller manager 确定哪个Node合适，最后由API server指挥远程工作节点中的一个组件 Kubelet ，由 Kubelet 创建 pod：Kubelet 通过 cri-dockerd 操作 docker 创建 pod。
在这里插入图片描述
首先，将一个 K8s 比作一个集团，集团要做不同的项目，集群的各个角色就担任了不同的任务。

Node：就是各个负责工作的地方也就是工厂。
Kubelet：每一个工厂的负责人。
k-proxy：每一个工厂的门卫，当总部的人要来巡视工厂了，可以通过它来询问当前的项目是不是在这开展，不管有没有在它这开展它都能告诉领导该去哪里查看。
controller manager：决策者，决定项目由哪一个工厂来开展。
API server：秘书部，决策者的决策不会直接告诉工厂而是通过它来进行转达，同样地，工厂的情况也是通过它来转给决策者。
scheduler：调度者，调度项目的执行。
etcd：资料库，用于存放集团的资料。
每一个服务器中要有一个监工：kubelet，由它来负责监控整个服务器里面容器的监控状况
所有的沟通都是通过秘书（api-server）
所有的服务器都要装上运行时环境，可以是docker。
可以通过命令的形式来进行部署

在这里插入图片描述

2 环境部署

1 ip规划：

vim /etc/hosts

192.168.21.20 kubernetes-master.inspur.com kubernetes-master
192.168.21.21 kubernetes-node1.inspur.com kubernetes-node1
192.168.21.22 kubernetes-node2.inspur.com kubernetes-node2
192.168.21.23 kubernetes-node3.inspur.com kubernetes-node3
192.168.21.24 kubernetes-register.inspur.com kubernetes-register

[root@inspur ~]# ssh-keygen -t rsa
[root@inspur ~]# ls .ssh/
id_rsa  id_rsa.pub

[root@inspur ~]# for i in 20 21 22 23 24
> do
> ssh-copy-id root@192.168.21.$i
> done

[root@inspur ~]# for i in 20 21 22 23 24; do ssh root@192.168.21.$i "hostname"; done
inspur
inspur
inspur
inspur
inspur

[root@inspur ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.110.35.201 registrylibrary.com.cn
192.168.21.20 kubernetes-master.inspur.com kubernetes-master
192.168.21.21 kubernetes-node1.inspur.com kubernetes-node1
192.168.21.22 kubernetes-node2.inspur.com kubernetes-node2
192.168.21.23 kubernetes-node3.inspur.com kubernetes-node3
192.168.21.24 kubernetes-register.inspur.com kubernetes-register
[root@inspur ~]# hostnamectl set-hostname  kubernetes-master
[root@inspur ~]# exec /bin/bash
[root@kubernetes-master ~]# hostname
kubernetes-master

[root@kubernetes-master ~]# ssh root@192.168.21.21 "hostnamectl set-hostname  kubernetes-node1"
[root@kubernetes-master ~]# ssh root@192.168.21.22 "hostnamectl set-hostname  kubernetes-node2"
[root@kubernetes-master ~]# ssh root@192.168.21.23 "hostnamectl set-hostname  kubernetes-node3"
[root@kubernetes-master ~]# ssh root@192.168.21.24 "hostnamectl set-hostname  kubernetes-register"

[root@kubernetes-master ~]# for i in 20 21 22 23 24; do ssh root@192.168.21.$i "hostname"; done
kubernetes-master
kubernetes-node1
kubernetes-node2
kubernetes-node3
kubernetes-register

2 不重启电脑，禁用启用swap，立刻生效

# 禁用命令

 sudo swapoff -a
# 启用命令

 sudo swapon -a
# 查看交换分区的状态

 sudo free -m

[root@kubernetes-master ~]# vim  /etc/sysctl.d/k8s.conf
[root@kubernetes-master ~]# cat /etc/sysctl.d/k8s.conf
vm.swappiness=0

3 网络参数调整

配置iptables参数，使得流经网桥的流量也经过iptables/netfilter防火墙

[root@kubernetes-master ~]# vim /etc/sysctl.d/k8s.conf
[root@kubernetes-master ~]# cat /etc/sysctl.d/k8s.conf
vm.swappiness=0
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1

配置生效

[root@kubernetes-master ~]# sudo modprobe overlay
[root@kubernetes-master ~]# sudo modprobe br_netfilter
[root@inspur ~]# sysctl -p /etc/sysctl.d/k8s.conf

4 容器环境配置

https://developer.aliyun.com/mirror/docker-ce?spm=a2c6h.13651102.0.0.3e221b11ixbXRM
dokcer的安装见之前的文章

systemctl enable docker

在这里插入图片描述
配置docker镜像加速器：

[root@kubernetes-master docker]# pwd
/etc/docker
[root@kubernetes-master docker]# cat daemon.json
{
  "registry-mirrors": ["https://4zmn196h.mirror.aliyuncs.com",
      "http://74f21445.m.daocloud.io",
      "https://registry.docker-cn.com",
      "http://hub-mirror.c.163.com",
      "https://docker.mirrors.ustc.edu.cn"
  ],
  "dns": ["8.8.8.8", "114.114.114.114"],
  "insecure-registries": ["kubernetes-register.inspur.com"],
  "exec-opts":[ "native.cgroupdriver=systemd"]
}
[root@kubernetes-master docker]#

5 cri环境操作

[root@kubernetes-master softs]# pwd
/data/softs
[root@kubernetes-master softs]# ll
total 0
[root@kubernetes-master softs]#

下载软件

[root@kubernetes-master softs]# wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.3.2/cri-dockerd-0.3.2.amd64.tgz

解压软件：

[root@kubernetes-master softs]# tar -zxvf cri-dockerd-0.3.2.amd64.tgz

[root@kubernetes-master system]# cd /usr/lib/systemd/system/
[root@kubernetes-master system]# chmod 777 cri-dockerd

在这里插入图片描述
配置启动文件：
/etc/systemd/system

[Unit]
Description=CRI Interface for Docker Application Container Engine
Documentation=https://docs.mirantis.com
After=network-online.target firewalld.service docker.service
Wants=network-online.target
[Service]
Type=notify
ExecStart=/usr/local/bin/cri-dockerd --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.9
--network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock --cri-dockerd-root-directory=/var/lib/dockershim --docker-endpoint=unix:///var/run/docker.sock --cri-dockerd-root-directory=/var/lib/docker
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s 
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target

生成socket文件：
/etc/systemd/system

[Unit]
Description=CRI Docker socket for the API
PartOf=cri-docker.service
[Socket]
ListenStream=/var/run/cri-dockerd.sock
SocketMode=0660
SocketUser=root
SocketGroup=docker
[Install]
WantedBy=sockets.target

[root@kubernetes-master system]# cd /etc/systemd/system/

在这里插入图片描述

在这里插入图片描述

检查效果：

[root@kubernetes-master bin]# systemctl status cri-dockerd

在这里插入图片描述

 systemctl is-active cri-dockerd

设置服务开机自启动：

[root@kubernetes-master bin]# systemctl daemon-reload
[root@kubernetes-master bin]# systemctl enable cri-dockerd.service
Created symlink from /etc/systemd/system/multi-user.target.wants/cri-dockerd.service to /etc/systemd/system/cri-dockerd.service.
[root@kubernetes-master bin]# systemctl restart cri-dockerd.service
[root@kubernetes-master bin]#

6 harbor仓库操作

下载软件：700M左右

wget https://github.com/goharbor/harbor/releases/download/v2.5.0/harbor-offline-installer-v2.5.0.tgz

curl -SL https://github.com/docker/compose/releases/download/v2.20.2/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose

mv docker-compose-linux-x86_64 /usr/local/bin/docker-compose

chmod +x docker-compose

sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

docker-compose --version

[root@kubernetes-register softs]# tar -zxvf docker-compose-linux-x86_64


[root@kubernetes-register server]# pwd
/data/server
[root@kubernetes-register server]# ll
total 0
drwxr-xr-x. 2 root root 122 Aug  7 15:57 harbor
[root@kubernetes-register server]# cd harbor/
[root@kubernetes-register harbor]# ll
total 647720
-rw-r--r--. 1 root root      3361 Apr  7  2022 common.sh
-rw-r--r--. 1 root root 663227387 Apr  7  2022 harbor.v2.5.0.tar.gz
-rw-r--r--. 1 root root      9917 Apr  7  2022 harbor.yml.tmpl
-rwxr-xr-x. 1 root root      2500 Apr  7  2022 install.sh
-rw-r--r--. 1 root root     11347 Apr  7  2022 LICENSE
-rwxr-xr-x. 1 root root      1881 Apr  7  2022 prepare
[root@kubernetes-register harbor]#

[root@kubernetes-register harbor]# docker load < harbor.v2.5.0.tar.gz

在这里插入图片描述
修改配置文件：

[root@kubernetes-register harbor]# cp harbor.yml.tmpl harbor.yml
[root@kubernetes-register harbor]# vim harbor.yml

hostname: kubernetes-register.inspur.com

禁用https:

#https:
  # https port for harbor, default is 443
 # port: 443
  # The path of cert and key files for nginx
  #certificate: /your/certificate/path
  #private_key: /your/private/key/path

harbor_admin_password: 123456

data_volume: /data/server/harbor/data

生成配置文件：

[root@kubernetes-register harbor]# ./prepare
prepare base dir is set to /data/server/harbor
WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /data/secret/keys/secretkey
Successfully called func: create_root_cert
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir

启动安装：

[root@kubernetes-register harbor]# ./install.sh

在这里插入图片描述


[root@kubernetes-register harbor]# docker compose ps
NAME                IMAGE                                COMMAND                  SERVICE             CREATED             STATUS                             PORTS
harbor-core         goharbor/harbor-core:v2.5.0          "/harbor/entrypoint.…"   core                5 minutes ago       Up 31 seconds (healthy)
harbor-db           goharbor/harbor-db:v2.5.0            "/docker-entrypoint.…"   postgresql          5 minutes ago       Up 32 seconds (healthy)
harbor-jobservice   goharbor/harbor-jobservice:v2.5.0    "/harbor/entrypoint.…"   jobservice          5 minutes ago       Up 29 seconds (health: starting)
harbor-log          goharbor/harbor-log:v2.5.0           "/bin/sh -c /usr/loc…"   log                 5 minutes ago       Up 4 minutes (healthy)             127.0.0.1:1514->10514/tcp
harbor-portal       goharbor/harbor-portal:v2.5.0        "nginx -g 'daemon of…"   portal              5 minutes ago       Up 32 seconds (healthy)
nginx               goharbor/nginx-photon:v2.5.0         "nginx -g 'daemon of…"   proxy               5 minutes ago       Up 29 seconds (health: starting)   0.0.0.0:80->8080/tcp, :::80->8080/tcp
redis               goharbor/redis-photon:v2.5.0         "redis-server /etc/r…"   redis               5 minutes ago       Up 32 seconds (healthy)
registry            goharbor/registry-photon:v2.5.0      "/home/harbor/entryp…"   registry            5 minutes ago       Up 32 seconds (healthy)
registryctl         goharbor/harbor-registryctl:v2.5.0   "/home/harbor/start.…"   registryctl         5 minutes ago       Up 32 seconds (healthy)
[root@kubernetes-register harbor]#

有一个问题：
在这里插入图片描述
下线处理：

[root@kubernetes-register harbor]# docker-compose down
[+] Running 10/10
 ✔ Container harbor-jobservice  Removed                                                                                                                                                      2.1s
 ✔ Container nginx              Removed                                                                                                                                                      2.1s
 ✔ Container registryctl        Removed                                                                                                                                                     10.4s
 ✔ Container harbor-portal      Removed                                                                                                                                                      0.6s
 ✔ Container harbor-core        Removed                                                                                                                                                      0.6s
 ✔ Container registry           Removed                                                                                                                                                      0.4s
 ✔ Container redis              Removed                                                                                                                                                      0.5s
 ✔ Container harbor-db          Removed                                                                                                                                                      0.5s
 ✔ Container harbor-log         Removed                                                                                                                                                     10.1s
 ✔ Network harbor_harbor        Removed

定制服务启动文件：

[root@kubernetes-register system]# pwd
/etc/systemd/system

[Unit]
Description=Harbor
After=docker.service systemd-networkd.service systemd-resolved.service
Requires=docker.service
Documentation=http://github.com/vmware/harbor
[Service]
Type=simple
Restart=on-failure
RestartSec=5
ExecStart=/usr/bin/docker-compose --file /data/server/harbor/docker-compose.yml upExecStop=/usr/bin/docker-compose --file /data/server/harbor/docker-compose.yml down
[Install]
WantedBy=multi-user.target

在这里插入图片描述

[root@kubernetes-register ~]# cd /data/server/harbor/
[root@kubernetes-register harbor]# docker-compose ps
NAME                IMAGE                                COMMAND                  SERVICE             CREATED             STATUS                   PORTS
harbor-core         goharbor/harbor-core:v2.5.0          "/harbor/entrypoint.…"   core                2 minutes ago       Up 2 minutes (healthy)
harbor-db           goharbor/harbor-db:v2.5.0            "/docker-entrypoint.…"   postgresql          2 minutes ago       Up 2 minutes (healthy)
harbor-jobservice   goharbor/harbor-jobservice:v2.5.0    "/harbor/entrypoint.…"   jobservice          2 minutes ago       Up 2 minutes (healthy)
harbor-log          goharbor/harbor-log:v2.5.0           "/bin/sh -c /usr/loc…"   log                 2 minutes ago       Up 2 minutes (healthy)   127.0.0.1:1514->10514/tcp
harbor-portal       goharbor/harbor-portal:v2.5.0        "nginx -g 'daemon of…"   portal              2 minutes ago       Up 2 minutes (healthy)
nginx               goharbor/nginx-photon:v2.5.0         "nginx -g 'daemon of…"   proxy               2 minutes ago       Up 2 minutes (healthy)   0.0.0.0:80->8080/tcp, :::80->8080/tcp
redis               goharbor/redis-photon:v2.5.0         "redis-server /etc/r…"   redis               2 minutes ago       Up 2 minutes (healthy)
registry            goharbor/registry-photon:v2.5.0      "/home/harbor/entryp…"   registry            2 minutes ago       Up 2 minutes (healthy)
registryctl         goharbor/harbor-registryctl:v2.5.0   "/home/harbor/start.…"   registryctl         2 minutes ago       Up 2 minutes (healthy)
[root@kubernetes-register harbor]#

创建一个用户：

Inspur930312043@qq.com

在这里插入图片描述
用自己的账号登录并创建一个项目：

[root@kubernetes-master ~]# docker tag  nginx:latest kubernetes-register.inspur.com/inspur/nginx:2.22.01


[root@kubernetes-master ~]# docker tag  tomcat:latest kubernetes-register.inspur.com/inspur/tomcat:11.1.01

上传镜像到私有仓库：

[root@kubernetes-master ~]# docker images
REPOSITORY                                     TAG       IMAGE ID       CREATED         SIZE
nginx                                          latest    605c77e624dd   19 months ago   141MB
kubernetes-register.inspur.com/inspur/nginx    2.22.01   605c77e624dd   19 months ago   141MB
tomcat                                         latest    fb5657adc892   19 months ago   680MB
kubernetes-register.inspur.com/inspur/tomcat   11.1.01   fb5657adc892   19 months ago   680MB
[root@kubernetes-master ~]# docker login kubernetes-register.inspur.com
Username: inspur
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[root@kubernetes-master ~]# docker push kubernetes-register.inspur.com/inspur/tomcat:11.1.01
The push refers to repository [kubernetes-register.inspur.com/inspur/tomcat]
3e2ed6847c7a: Pushed
bd2befca2f7e: Pushed
59c516e5b6fa: Pushed
3bb5258f46d2: Pushed
832e177bb500: Pushed
f9e18e59a565: Pushed
26a504e63be4: Pushed
8bf42db0de72: Pushed
31892cc314cb: Pushed
11936051f93b: Pushed
11.1.01: digest: sha256:e6d65986e3b0320bebd85733be1195179dbce481201a6b3c1ed27510cfa18351 size: 2422
[root@kubernetes-master ~]# docker push kubernetes-register.inspur.com/inspur/nginx:2.22.01
The push refers to repository [kubernetes-register.inspur.com/inspur/nginx]
d874fd2bc83b: Pushed
32ce5f6a5106: Pushed
f1db227348d0: Pushed
b8d6e692a25e: Pushed
e379e8aedd4d: Pushed
2edcec3590a4: Pushed
2.22.01: digest: sha256:ee89b00528ff4f02f2405e4ee221743ebc3f8e8dd0bfd5c4c20a2fa2aaa7ede3 size: 1570
[root@kubernetes-master ~]#

拉取镜像：

[root@kubernetes-register harbor]# docker pull kubernetes-register.inspur.com/inspur/tomcat@sha256:e6d65986e3b0320bebd85733be1195179dbce481201a6b3c1ed27510cfa18351
kubernetes-register.inspur.com/inspur/tomcat@sha256:e6d65986e3b0320bebd85733be1195179dbce481201a6b3c1ed27510cfa18351: Pulling from inspur/tomcat
Digest: sha256:e6d65986e3b0320bebd85733be1195179dbce481201a6b3c1ed27510cfa18351
Status: Image is up to date for kubernetes-register.inspur.com/inspur/tomcat@sha256:e6d65986e3b0320bebd85733be1195179dbce481201a6b3c1ed27510cfa18351
kubernetes-register.inspur.com/inspur/tomcat@sha256:e6d65986e3b0320bebd85733be1195179dbce481201a6b3c1ed27510cfa18351
[root@kubernetes-register harbor]# docker images
REPOSITORY                                     TAG       IMAGE ID       CREATED         SIZE
goharbor/harbor-exporter                       v2.5.0    36396f138dfb   16 months ago   86.7MB
goharbor/chartmuseum-photon                    v2.5.0    eaedcf1f700b   16 months ago   225MB
goharbor/redis-photon                          v2.5.0    1e00fcc9ae63   16 months ago   156MB
goharbor/trivy-adapter-photon                  v2.5.0    4e24a6327c97   16 months ago   164MB
goharbor/notary-server-photon                  v2.5.0    6d5fe726af7f   16 months ago   112MB
goharbor/notary-signer-photon                  v2.5.0    932eed8b6e8d   16 months ago   109MB
goharbor/harbor-registryctl                    v2.5.0    90ef6b10ab31   16 months ago   136MB
goharbor/registry-photon                       v2.5.0    30e130148067   16 months ago   77.5MB
goharbor/nginx-photon                          v2.5.0    5041274b8b8a   16 months ago   44MB
goharbor/harbor-log                            v2.5.0    89fd73f9714d   16 months ago   160MB
goharbor/harbor-jobservice                     v2.5.0    1d097e877be4   16 months ago   226MB
goharbor/harbor-core                           v2.5.0    42a54bc05b02   16 months ago   202MB
goharbor/harbor-portal                         v2.5.0    c206e936f4f9   16 months ago   52.3MB
goharbor/harbor-db                             v2.5.0    d40a1ae87646   16 months ago   223MB
goharbor/prepare                               v2.5.0    36539574668f   16 months ago   268MB
tomcat                                         latest    fb5657adc892   19 months ago   680MB
kubernetes-register.inspur.com/inspur/tomcat   11.1.01   fb5657adc892   19 months ago   680MB

7 K8s集群初始化

在这里插入图片描述
配置阿里云镜像：

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
setenforce 0

软件安装：

yum install -y kubelet kubeadm kubectl
systemctl enable kubelet && systemctl start kubelet

在这里插入图片描述

[root@kubernetes-master ~]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.4", GitCommit:"fa3d7990104d7c1f16943a67f11b154b71f6a132", GitTreeState:"clean", BuildDate:"2023-07-19T12:19:40Z", GoVersion:"go1.20.6", Compiler:"gc", Platform:"linux/amd64"}
[root@kubernetes-master ~]# kubeadm config images list
registry.k8s.io/kube-apiserver:v1.27.4
registry.k8s.io/kube-controller-manager:v1.27.4
registry.k8s.io/kube-scheduler:v1.27.4
registry.k8s.io/kube-proxy:v1.27.4
registry.k8s.io/pause:3.9
registry.k8s.io/etcd:3.5.7-0
registry.k8s.io/coredns/coredns:v1.10.1
[root@kubernetes-master ~]# kubeadm config list | awk -F '/' '{print $NF}'
invalid subcommand "list"
See 'kubeadm config -h' for help and examples
[root@kubernetes-master ~]# kubeadm config images list | awk -F '/' '{print $NF}'
kube-apiserver:v1.27.4
kube-controller-manager:v1.27.4
kube-scheduler:v1.27.4
kube-proxy:v1.27.4
pause:3.9
etcd:3.5.7-0
coredns:v1.10.1

images=$(kubeadm config images list --kubernetes-version=1.27.4 | awk -F "/" '{print $NF}')
for i in ${images}
do
docker pull registry.aliyuncs.com/google_containers/$i
docker tag registry.aliyuncs.com/google_containers/$i kubernetes-register.inspur.com/google_containers/$i
docker push kubernetes-register.inspur.com/google_containers/$i
docker rmi registry.aliyuncs.com/google_containers/$i
done

环境初始化：
在这里插入图片描述

[root@kubernetes-master ~]# kubeadm init --kubernetes-version=1.27.4 --apiserver-advertise-address=192.168.21.20 --image-repository=kubernetes-register.inspur.com/google_containers --pod-network-cidr="172.17.1.0/16" --service-cidr="10.96.0.0/12" --ignore-preflight-errors=Swap --cri-socket=unix:///var/run/cri-dockerd.sock

分别在node1 node2 node3 执行

kubeadm join 192.168.21.20:6443 --token ymlg0w.xkl1z4pr7dwg39yr --discovery-token-ca-cert-hash sha256:ec15cba4c2e6bf2291ff1217f4e9bd41ee28aa02139e6fc86e614a5b8c6f4b96 --cri-socket=unix:///var/run/cri-dockerd.sock

但是现在还是有一个问题：
在这里插入图片描述
运行如下命令：

[root@kubernetes-master ~]#   mkdir -p $HOME/.kube
[root@kubernetes-master ~]#   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@kubernetes-master ~]#   sudo chown $(id -u):$(id -g) $HOME/.kube/config

[root@kubernetes-master ~]# kubectl get nodes
NAME                STATUS     ROLES           AGE     VERSION
kubernetes-master   NotReady   control-plane   5m8s    v1.27.4
kubernetes-node1    NotReady   <none>          2m56s   v1.27.4
kubernetes-node2    NotReady   <none>          2m46s   v1.27.4
kubernetes-node3    NotReady   <none>          2m44s   v1.27.4
[root@kubernetes-master ~]#

命令补全：

source <(kubectl completion bash)
source <(kubeadm completion bash)

[root@kubernetes-master ~]# vim .bashrc
[root@kubernetes-master ~]# cat .bashrc
# .bashrc

# User specific aliases and functions

alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'

# Source global definitions
if [ -f /etc/bashrc ]; then
        . /etc/bashrc
fi
source <(kubectl completion bash)
source <(kubeadm completion bash)
[root@kubernetes-master ~]# source .bashrc
[root@kubernetes-master ~]# kubectl get n
namespaces                         networkpolicies.networking.k8s.io  nodes
[root@kubernetes-master ~]# kubectl get n
namespaces                         networkpolicies.networking.k8s.io  nodes
[root@kubernetes-master ~]# kubectl get n

网络配置：

 https://kubernetes.io/docs/concepts/cluster-administration/addons/


[root@kubernetes-master flannel]# pwd
/data/kubernetes/network/flannel
[root@kubernetes-master flannel]# ll
total 16
-rw-r--r--. 1 root root 4459 Aug  8 09:42 kube-flannel.yml
-rw-r--r--. 1 root root 4459 Aug  8 09:42 kube-flannel.ymlBK
[root@kubernetes-master flannel]#

在这里插入图片描述
需要下载一下三个镜像文件：

[root@kubernetes-master flannel]# grep image: kube-flannel.yml
        image: docker.io/flannel/flannel:v0.22.1
        image: docker.io/flannel/flannel-cni-plugin:v1.2.0
        image: docker.io/flannel/flannel:v0.22.1

[root@kubernetes-master flannel]# docker tag flannel/flannel:v0.22.1 kubernetes-register.inspur.com/inspur/flannel/flannel:v0.22.1
[root@kubernetes-master flannel]# docker tag flannel/flannel-cni-plugin:v1.2.0 kubernetes-register.inspur.com/inspur/flannel/flannel-cni-plugin:v1.2.0

推送到远程私有仓库：

[root@kubernetes-master flannel]# docker push  kubernetes-register.inspur.com/inspur/flannel/flannel:v0.22.1
The push refers to repository [kubernetes-register.inspur.com/inspur/flannel/flannel]
c120dc707e2b: Pushed
92e727b491c6: Pushed
23d8410e16ff: Pushed
9726974aad9b: Pushed
117eda391951: Pushed
40afc0fa6094: Pushed
d8ddfa4843bd: Pushed
f1417ff83b31: Pushed
v0.22.1: digest: sha256:02cb1205742b0d087f89d654cbe5fa8f815e97e4e5cc86c7f39bd7933d1a04cb size: 1996
[root@kubernetes-master flannel]# docker push kubernetes-register.inspur.com/inspur/flannel/flannel-cni-plugin:v1.2.0
The push refers to repository [kubernetes-register.inspur.com/inspur/flannel/flannel-cni-plugin]
9332f71f5f3d: Pushed
7df5bd7bd262: Pushed
v1.2.0: digest: sha256:2180bb74f60bea56da2e9be2004271baa6dccc0960b7aeaf43a97fc4de9b1ae0 size: 739
[root@kubernetes-master flannel]#

修改yml文件：
在这里插入图片描述

[root@kubernetes-master flannel]# kubectl apply -f kube-flannel.yml
namespace/kube-flannel created
serviceaccount/flannel created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
[root@kubernetes-master flannel]#

查看namespace：

[root@kubernetes-master flannel]# kubectl get ns
NAME              STATUS   AGE
default           Active   34m
kube-flannel      Active   53s
kube-node-lease   Active   34m
kube-public       Active   34m
kube-system       Active   34m
[root@kubernetes-master flannel]#

[root@kubernetes-master flannel]# kubectl get pod -n kube-flannel
NAME                    READY   STATUS             RESTARTS      AGE
kube-flannel-ds-mdpkb   0/1     CrashLoopBackOff   4 (29s ago)   2m33s
kube-flannel-ds-pg4x6   0/1     CrashLoopBackOff   4 (44s ago)   2m33s
kube-flannel-ds-vjw5j   0/1     CrashLoopBackOff   4 (40s ago)   2m33s
kube-flannel-ds-wzwx9   0/1     CrashLoopBackOff   4 (41s ago)   2m33s
[root@kubernetes-master flannel]# kubectl get nodes
NAME                STATUS   ROLES           AGE   VERSION
kubernetes-master   Ready    control-plane   36m   v1.27.4
kubernetes-node1    Ready    <none>          34m   v1.27.4
kubernetes-node2    Ready    <none>          34m   v1.27.4
kubernetes-node3    Ready    <none>          34m   v1.27.4
[root@kubernetes-master flannel]#

在这里插入图片描述

[root@kubernetes-master flannel]# systemctl is-active kubelet cri-dockerd docker
active
active
active
[root@kubernetes-master flannel]# systemctl enable  kubelet cri-dockerd docker