前言：本章主要记录ingress暴露服务方式，会详细解释其原理及两种网络模式应用实操。
1、Ingress概念详解
Kubernetes 暴露服务的方式目前只有三种：LoadBlancer Service、NodePort Service、Ingress，Service属于4层负载均衡，而本章的ingress属于7层负载均衡。
要理解 Ingress，需要区分两个概念：Ingress（类似老板角色） 和 Ingress-Controller（类似职工角色）。
Ingress 对象： 这是 Kubernetes 中的一个 API 对象/资源对象，用 YAML文件 配置。它定义了请求如何转发到 Service 的规则，可以理解为配置模板。
Ingress-Controller： 这是具体实现反向代理和负载均衡的程序。它解析 Ingress 定义的规则，并根据配置转发请求。
简而言之，Ingress-Controller 负责实际的请求转发，并通过多种方式暴露在集群入口。外部请求首先到达 Ingress-Controller，而 Ingress 对象则告知 Ingress-Controller 如何转发请求，如哪些域名和路径转发到哪些服务。
Service 在第四层传输层（TCP/IP 层）以 IP:PORT 形式存在，对于不同的 URL 地址，可能需要不同的后端服务或虚拟服务器。这种应用层的转发机制通过 Kubernetes 的 Service 机制无法实现，此时可以使用 Ingress 策略和具体的 Ingress-Controller。Ingress 提供七层负载均衡能力（当然也无法脱离4层存在），如外部可访问的 URL、负载均衡、SSL、基于名称的虚拟主机等。在集群流量接入层，Ingress 的高可用性至关重要。
Kubernetes 并没有自带 Ingress Controller，实际上ingress-controller只是一个统称，具体实现有多种，需要自己单独安装，目前，由k8s维护的ingress-controller只有google云的GCE与ingress-nginx两个，常用的是 Ingress-nginx Controller（基于nginx七层反向代理来实现）。
Ingress 一般由三个组件组成：
- Nginx 反向代理负载均衡器（干活用的工具）：负责实际的请求转发和负载均衡。
- Ingress Controller（员工）：作为控制器，与 Kubernetes API 交互，实时获取后端 Service 和 Pod 的变化（如新增或删除）。它结合 Ingress 定义的规则生成配置，动态更新 Nginx 负载均衡器，并使配置生效，实现服务自动发现。
- Ingress（老板）：定义请求转发规则。通过 YAML 文件配置，指定某个域名的请求如何转发到集群中的指定 Service。可以为一个或多个 Service 定义一个或多个 Ingress 规则。
2、Ingress部署实战
2.1、部署 Ingress-Controller（Nginx）
下载Ingress控制器：
网盘链接: https://pan.baidu.com/s/1Y3eB-4OXeoOQp1AVoaXdYw?pwd=tfdm 提取码: tfdm
github地址：https://github.com/kubernetes/ingress-nginx/archive/nginx-0.30.0.tar.gz
[root@k8s-master1 ~]# wget https://github.com/kubernetes/ingress-nginx/archive/nginx-0.30.0.tar.gz
[root@k8s-master1 ~]# tar -zxvf ingress-nginx-nginx-0.30.0.tar.gz -C /mnt/
[root@k8s-master1 ~]# cd /mnt/ingress-nginx-nginx-0.30.0/deploy/static/
[root@k8s-master1 static]# ls
configmap.yaml mandatory.yaml namespace.yaml provider rbac.yaml with-rbac.yaml
[root@k8s-master1 static]# mkdir /root/ingress
[root@k8s-master1 static]# cp mandatory.yaml /root/ingress
[root@k8s-master1 ~]# cd /root/ingress
[root@k8s-master1 ~]# cd /root/ingress
[root@k8s-master1 ingress]# vim mandatory.yaml
修改详情如下（修改后的文件网盘连接内有）1、修改pod控制器为daemonset
apiVersion: apps/v1
kind: DaemonSet #修改
metadata:
name: nginx-ingress-controller
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
#replicas: 1 #注释掉
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
2、宿主机网络共享，注释掉node标签选择
spec:
# wait up to five minutes for the drain of connections
terminationGracePeriodSeconds: 300
hostNetwork: true #添加宿主机网络共享
serviceAccountName: nginx-ingress-serviceaccount
#nodeSelector:
#kubernetes.io/os: linux #这两行注释掉
这里不注释的话需要使用
kubectl label nodes k8s-node1 kubernetes.io/os: linux=true为node节点添加标签
3、修改镜像为阿里镜像（直接拉取也可以，但网络不稳定，成功拉取后记得上传到本地仓库）
#image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
image: registry.cn-chengdu.aliyuncs.com/k8s_module_images/kubernetes-ingress-controller-nginx-ingress-controller:0.30.0
[root@k8s-master1 ingress]# kubectl apply -f mandatory.yaml
此处一般会有警告提示修改api版本，忽略的话不影响后续使用
[root@k8s-master1 ingress]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-f6brt 1/1 Running 0 2m30s
nginx-ingress-controller-jbst2 1/1 Running 0 2m30s此时Ingress-Controller部署完毕
2.2、部署web应用
这里选用http和nginx两个作为两种不同的web服务器，这里使用hostnetwork网络方式，所以部署的svc是默认的clusterip
[root@k8s-master1 ingress]# vim my-apache.yaml
[root@k8s-master1 ingress]# cat my-apache.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-apache
spec:
selector:
matchLabels:
run: my-apache
replicas: 2
template:
metadata:
labels:
run: my-apache
spec:
containers:
- name: my-apache
image: registry.cn-chengdu.aliyuncs.com/liumuquan_app/httpd:2.4-alpine3.17
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: my-apache
labels:
run: my-apache
spec:
#type: NodePort
ports:
- port: 80
targetPort: 80
#nodePort: 30001
selector:
run: my-apache
[root@k8s-master1 ingress]# vim my-nginx.yaml
[root@k8s-master1 ingress]# cat my-nginx.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
selector:
matchLabels:
run: my-nginx
replicas: 2
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: registry.cn-chengdu.aliyuncs.com/liumuquan_app/nginx:1.12.0-alpine
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: my-nginx
labels:
run: my-nginx
spec:
#type: NodePort
ports:
- port: 80
targetPort: 80
#nodePort: 30002
selector:
run: my-nginx创建两个web服务
[root@k8s-master1 ingress]# kubectl apply -f my-apache.yaml
deployment.apps/my-apache created
service/my-apache created
[root@k8s-master1 ingress]# kubectl apply -f my-nginx.yaml
deployment.apps/my-nginx created
service/my-nginx created
[root@k8s-master1 ingress]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
my-apache-764b8f49c5-g6hl6 1/1 Running 0 23s 10.244.1.60 k8s-node1 <none> <none>
my-apache-764b8f49c5-t5ndt 1/1 Running 0 23s 10.244.2.45 k8s-node2 <none> <none>
my-nginx-5c49645686-rm6h8 1/1 Running 0 19s 10.244.1.61 k8s-node1 <none> <none>
my-nginx-5c49645686-z7g82 1/1 Running 0 19s 10.244.2.46 k8s-node2 <none> <none>
[root@k8s-master1 ingress]# kubectl get endpoints
NAME ENDPOINTS AGE
cluster.local-nfs-redis-nfs-client-provisioner <none> 2d23h
kubernetes 192.168.188.101:6443 23d
my-apache 10.244.1.60:80,10.244.2.45:80 34s
my-nginx 10.244.1.61:80,10.244.2.46:80 30s
[root@k8s-master1 ingress]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 23d
my-apache ClusterIP 10.96.232.117 <none> 80/TCP 39s
my-nginx ClusterIP 10.99.230.106 <none> 80/TCP 35s
web服务创建完成
2.3、配置ingress规则
1）宿主机网络模式 hostNetwork: true
[root@k8s-master1 ingress]# vim ingress-test.yaml
[root@k8s-master1 ingress]# cat ingress-test.yaml
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
namespace: default
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules: #定义转发规则
- host: www.apache111.com #指定域名方式
http:
paths:
- path: / #指定访问的路径
pathType: Prefix #定义路径的类型
backend: #定义转发后端的服务
service: #定义转发的service
name: my-apache
port:
number: 80 #由于Ingress控制器开启了hostNetwork: true。且clusterip的配置这里必须设置80
- host: www.nginx222.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-nginx
port:
number: 80[root@k8s-master1 ingress]# kubectl apply -f ingress-test.yaml
ingress.networking.k8s.io/test-ingress created
[root@k8s-master1 ingress]# kubectl get ingress
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
test-ingress <none> www.apache111.com,www.nginx222.com 80 8s配置物理机解析
windows电脑：
C:\Windows\System32\drivers\etc\hosts内添加
192.168.188.102 k8s-node1 www.apache111.com www.nginx222.com
192.168.188.103 k8s-node2 www.apache111.com www.nginx222.com使用物理机浏览器访问
2）修改网络模式为nodeport
[root@k8s-master1 ingress]# kubectl delete -f ingress-test.yaml
[root@k8s-master1 ingress]# kubectl delete -f my-apache.yaml
[root@k8s-master1 ingress]# kubectl delete -f my-nginx.yaml
[root@k8s-master1 ingress]# kubectl delete -f mandatory.yamlingress-controller配置如下
[root@k8s-master1 ingress]# vim ingress-controller.yaml
[root@k8s-master1 ingress]# cat ingress-controller.yaml
---
apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: tcp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: udp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: nginx-ingress-clusterrole
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- "extensions"
- "networking.k8s.io"
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
- "networking.k8s.io"
resources:
- ingresses/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: nginx-ingress-role
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
# Defaults to "<election-id>-<ingress-class>"
# Here: "<ingress-controller-leader>-<nginx>"
# This has to be adapted if you change either parameter
# when launching the nginx-ingress-controller.
- "ingress-controller-leader-nginx"
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: nginx-ingress-role-nisa-binding
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nginx-ingress-role
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nginx-ingress-clusterrole-nisa-binding
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-ingress-clusterrole
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: apps/v1
#kind: DaemonSet
kind: Deployment
metadata:
name: nginx-ingress-controller
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
replicas: 2
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
spec:
# wait up to five minutes for the drain of connections
terminationGracePeriodSeconds: 300
# hostNetwork: true # 关闭此网络模式
serviceAccountName: nginx-ingress-serviceaccount
# nodeSelector:
# custom/ingress-controller-ready: "true"
# kubernetes.io/os: linux
containers:
- name: nginx-ingress-controller
image: registry.cn-chengdu.aliyuncs.com/k8s_module_images/kubernetes-ingress-controller-nginx-ingress-controller:0.30.0
args:
- /nginx-ingress-controller
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --publish-service=$(POD_NAMESPACE)/ingress-nginx
- --annotations-prefix=nginx.ingress.kubernetes.io
securityContext:
allowPrivilegeEscalation: true
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
# www-data -> 101
runAsUser: 101
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- name: http
containerPort: 80
protocol: TCP
- name: https
containerPort: 443
protocol: TCP
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
---
apiVersion: v1
kind: LimitRange
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
limits:
- min:
memory: 90Mi
cpu: 100m
type: Container创建ingress-controller
[root@k8s-master1 ingress]# kubectl apply -f ingress-controller.yaml
[root@k8s-master1 ingress]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-7b5d6f74c7-sxm7l 1/1 Running 0 85s
nginx-ingress-controller-7b5d6f74c7-tqpgl 1/1 Running 0 85s修改web-pod文件svc为nodeport方式
[root@k8s-master1 ingress]# vim my-apache.yaml
[root@k8s-master1 ingress]# cat my-apache.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-apache
spec:
selector:
matchLabels:
run: my-apache
replicas: 2
template:
metadata:
labels:
run: my-apache
spec:
containers:
- name: my-apache
image: registry.cn-chengdu.aliyuncs.com/liumuquan_app/httpd:2.4-alpine3.17
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: my-apache
labels:
run: my-apache
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
nodePort: 30001
selector:
run: my-apache
[root@k8s-master1 ingress]# vim my-nginx.yaml
[root@k8s-master1 ingress]# cat my-nginx.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
selector:
matchLabels:
run: my-nginx
replicas: 2
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: registry.cn-chengdu.aliyuncs.com/liumuquan_app/nginx:1.12.0-alpine
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: my-nginx
labels:
run: my-nginx
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
nodePort: 30002
selector:
run: my-nginx
创建webpod和svc
[root@k8s-master1 ingress]# kubectl apply -f my-apache.yaml
deployment.apps/my-apache created
service/my-apache created
[root@k8s-master1 ingress]# kubectl apply -f my-nginx.yaml
deployment.apps/my-nginx created
service/my-nginx created
[root@k8s-master1 ingress]# kubectl get pod
NAME READY STATUS RESTARTS AGE
my-apache-764b8f49c5-5bgdb 1/1 Running 0 19s
my-apache-764b8f49c5-ck4vl 1/1 Running 0 19s
my-nginx-5c49645686-7csv8 1/1 Running 0 15s
my-nginx-5c49645686-ptxhv 1/1 Running 0 15s
[root@k8s-master1 ingress]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 23d
my-apache NodePort 10.105.128.125 <none> 80:30001/TCP 22s
my-nginx NodePort 10.99.184.224 <none> 80:30002/TCP 18s
ingress无需修改
[root@k8s-master1 ingress]# kubectl apply -f ingress-test.yaml
ingress.networking.k8s.io/test-ingress created此时物理机访问需要添加nodeport暴露的端口号才能正常访问
