一、主机准备
主机硬件配置说明
| 作用 | IP地址 | 操作系统 | 配置 |
|---|---|---|---|
| k8s-master01 | 192.168.136.55 | openEuler-22.03-LTS-SP1 | 2颗CPU 4G内存 50G硬盘 |
| k8s-node01 | 192.168.136.56 | openEuler-22.03-LTS-SP1 | 2颗CPU 4G内存 50G硬盘 |
| k8s-node02 | 192.168.136.57 | openEuler-22.03-LTS-SP1 | 2颗CPU 4G内存 50G硬盘 |
1、配置主机名和IP
hostnamectl set-hostname k8s-master01
hostnamectl set-hostname k8s-node01
hostnamectl set-hostname k8s-node022、配置hosts解析
# cat >> /etc/hosts << EOF
192.168.136.55 k8s-master01
192.168.136.56 k8s-node01
192.168.136.57 k8s-node02
EOF3、配置免密登录，只在k8s-master01上操作
[root@k8s-master01 ~]# ssh-keygen -f ~/.ssh/id_rsa -N '' -q
# 点拷贝秘钥到其他 2 台节点
[root@k8s-master01 ~]# ssh-copy-id k8s-node01
[root@k8s-master01 ~]# ssh-copy-id k8s-node024、防火墙和SELinux
systemctl disable firewalld
systemctl stop firewalld
sed -ri 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
setenforce 05、时间同步配置
dnf install -y chrony
systemctl restart chronyd
systemctl enable chronyd6、配置内核转发及网桥过滤
添加网桥过滤及内核转发配置文件
# cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
vm.swappiness = 0
EOF
加载br_netfilter模块
# modprobe br_netfilter
查看是否加载
# lsmod | grep br_netfilter
br_netfilter 22256 0
bridge 151336 1 br_netfilter
使用新添加配置文件生效
# sysctl -p /etc/sysctl.d/k8s.conf7、启用ipvs
三台机器
cat > /etc/sysconfig/modules/ipvs.modules <<END
#!/bin/bash
ipvs_modules="ip_vs ip_vs_lc ip_vs_wlc ip_vs_rr ip_vs_wrr ip_vs_lblc ip_vs_lblcr ip_vs_dh ip_vs_sh ip_vs_nq ip_vs_sed ip_vs_ftp nf_conntrack"
for kernel_module in ${ipvs_modules}
do
/sbin/modinfo -F filename ${kernel_module} > /dev/null 2>&1
if [ 0 -eq 0 ]
then
/sbin/modprobe ${kernel_module}
fi
done
END
chmod 755 /etc/sysconfig/modules/ipvs.modules
bash /etc/sysconfig/modules/ipvs.modules8、关闭swap
临时关闭
# swapoff -a
永远关闭swap分区
sed -i 's/.*swap.*/#&/' /etc/fstab二、容器运行时工具安装及运行
1、安装docker
1、下载官方repo
cd /etc/yum.repos.d/
curl -O https://download.docker.com/linux/centos/docker-ce.repo
sed -i 's/$releasever/8/g' docker-ce.repo
2. 安装docker
yum install -y docker-ce
# 设置国内镜像加速
mkdir -p /etc/docker
tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": [
"https://hub-mirror.c.163.com",
"https://mirror.baidubce.com",
"https://ccr.ccs.tencentyun.com"
]
}
EOF
设置docker开机启动并启动
# systemctl enable --now docker
查看docker版本
# docker version
# 所有节点执行以下命令，将 docker 的 CgroupDriver 改成 systemd，在/etc/docker/daemon.json 中添加配置：
{
"exec-opts": ["native.cgroupdriver=systemd"],
"registry-mirrors": [
"https://hub-mirror.c.163.com",
"https://mirror.baidubce.com",
"https://ccr.ccs.tencentyun.com"
]
}
# 所有节点执行以下命令，重启 docker：
systemctl daemon-reload
systemctl restart docker因为k8s 1.25以后版本默认装的是containerd，所以还要安装cri-dockerd
2、安装cri-dockerd
1. 下载最新版cri-dockerd rpm包
网络条件好的话直接使用wget下载，网络条件一般的话可以在github上面先下载再上传到虚拟机
下载地址：Releases · Mirantis/cri-dockerd (github.com)。
https://github.com/Mirantis/cri-dockerd/releases/download/v0.3.4/cri-dockerd-0.3.4-3.el8.x86_64.rpm
2、安装cri-docker
# wget -c http://github.com/Mirantis/cri-dockerd/releases/download/v0.3.4/cri-dockerd-0.3.4-3.el8.x86_64.rpm
# yum install cri-dockerd-0.3.4-3.el8.x86_64.rpm3、启动cri-docker服务
systemctl enable cri-docker4、cri-dockerd设置国内镜像加速
$ vi /usr/lib/systemd/system/cri-docker.service # 找到第10行ExecStart=
# 修改为ExecStart=/usr/bin/cri-dockerd --network-plugin=cni --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.9
# 重启Docker组件
$ systemctl daemon-reload && systemctl restart docker cri-docker.socket cri-docker
# 检查Docker组件状态
$ systemctl status docker cir-docker.socket cri-docker三、K8S软件安装
1、配置kubernetes源
# 此操作会覆盖 /etc/yum.repos.d/kubernetes.repo 中现存的所有配置
cat <<EOF | tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.28/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.28/rpm/repodata/repomd.xml.key
#exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF
2、安装kubelet、kubeadm、kubectl、kubernetes-cni
yum install -y kubelet kubeadm kubectl kubernetes-cni
systemctl enable kubelet.service四、K8S集群初始化
只在master01节点上操作
创建初始化文件 kubeadm-init.yaml
[root@k8s-master01 ~]# kubeadm config print init-defaults > kubeadm-init.yaml修改如下配置：
- advertiseAddress：为控制平面地址，（ Master 主机 IP ）
advertiseAddress: 1.2.3.4
修改为 advertiseAddress: 192.168.136.55
- criSocket：为 containerd 的 socket 文件地址
criSocket: unix:///var/run/containerd/containerd.sock
修改为 criSocket: unix:///var/run/cri-dockerd.sock
- name: node 修改node为 k8s-master01
name: node
修改为 name: k8s-master01
高可用才添加下面的
clusterName下面添加 VIP和端口
controlPlaneEndpoint: 192.168.136.50:16443
- imageRepository：阿里云镜像代理地址，否则拉取镜像会失败
imageRepository: registry.k8s.io
修改为：imageRepository: registry.aliyuncs.com/google_containers
- kubernetesVersion：为 k8s 版本
kubernetesVersion: 1.28.0
修改为：kubernetesVersion: 1.28.9
注意：一定要配置镜像代理，否则会由于防火墙问题导致集群安装失败
文件末尾增加启用ipvs功能
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs
# 根据配置文件启动 kubeadm 初始化 k8s
$ kubeadm init --config=kubeadm-init.yaml --upload-certs --v=6注意：这里可能会报错
[ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
error execution phase preflight
解决：
[root@k8s-master01 ~]# sysctl -w net.ipv4.ip_forward=1
此页面 成功
[root@k8s-master01 ~]# mkdir -p $HOME/.kube
[root@k8s-master01 ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@k8s-master01 ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config
[root@k8s-master01 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master01 NotReady control-plane 3m48s v1.28.9
五、K8S集群工作节点加入
一定要加上 --cri-socket unix:///var/run/cri-dockerd.sock
[root@k8s-node01 ~]# kubeadm join 192.168.136.55:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:6f28419e1b194870cdae7a25803752d968eac71a96dcda865b1e5fbe267396df --cri-socket unix:///var/run/cri-dockerd.sock
六、K8S集群网络插件使用
# 只在master01上操作
[root@k8s-master01 ~]# curl -O https://docs.projectcalico.org/archive/v3.23/manifests/calico.yaml
[root@k8s-master01 ~]# kubectl create -f calico.yaml
[root@k8s-master01 ~]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-7cb4fd5784-7l5cs 1/1 Running 0 8m3s
calico-node-k5m7x 1/1 Running 0 8m3s
calico-node-mjjzs 1/1 Running 0 8m3s
calico-node-pzskl 1/1 Running 0 8m3s
coredns-66f779496c-767wf 1/1 Running 0 27m
coredns-66f779496c-ltw5p 1/1 Running 0 27m
etcd-k8s-master01 1/1 Running 0 27m
kube-apiserver-k8s-master01 1/1 Running 0 27m
kube-controller-manager-k8s-master01 1/1 Running 0 27m
kube-proxy-hmxvm 1/1 Running 0 24m
kube-proxy-htgbt 1/1 Running 0 24m
kube-proxy-tbm7d 1/1 Running 0 27m
kube-scheduler-k8s-master01 1/1 Running 0 27m
部署成功
