K8S中ingress详解

Ingress介绍

Kubernetes 集群中，服务（Service）是一种抽象，它定义了一种访问 Pod 的方式，无论这些 Pod 如何变化，服务都保持不变。服务可以被映射到一个静态的 IP 地址（ClusterIP）、一个 NodePort（在集群的每个节点上的特定端口）、一个 LoadBalancer（通过云服务提供商的负载均衡器）或一个外部 IP。
Service的两种服务暴露方式，NodePort 和 LoadBalancer，确存在一些局限性：
- NodePort：当一个服务被配置为 NodePort 类型时，它会在集群的所有节点上的一个静态端口上暴露服务。这种方式的缺点是，如果集群中有大量的服务，那么就需要占用大量的端口，而这些端口资源是有限的。
- LoadBalancer：这种方式通过云服务提供商的负载均衡器来暴露服务。虽然它解决了 NodePort 方式中端口资源有限的问题，但是每个服务都需要一个单独的负载均衡器，这不仅增加了成本，而且管理起来也相对复杂。
为了解决这些问题，Kubernetes 引入了 Ingress 资源对象：
- Ingress 是一种 API 对象，它管理外部访问到集群内服务的 HTTP 和 HTTPS 路由。它提供了一种规则，允许你将外部 HTTP/HTTPS 路由到集群内的多个服务。
- Ingress 可以提供单一的 IP 地址，通过不同的 URL 路径或不同的端口来路由到不同的服务。
- 它只需要一个 NodePort 或者一个 LoadBalancer，就可以将多个服务暴露给外部网络，这样做既节省了资源，又简化了配置。
- Ingress 还支持 SSL/TLS 终止，可以为不同的服务配置 SSL 证书。
- 它允许更复杂的路由规则，比如基于路径、主机名或 HTTP 头部的路由。

实际上，Ingress相当于一个7层的负载均衡器，是kubernetes对反向代理的一个抽象，它的工作原理类似于Nginx，可以理解成在Ingress里建立诸多映射规则，Ingress Controller通过监听这些配置规则并转化成Nginx的反向代理配置 , 然后对外部提供服务。在这里有两个核心概念：
- ingress：kubernetes中的一个对象，作用是定义请求如何转发到service的规则
- ingress controller：具体实现反向代理及负载均衡的程序，对ingress定义的规则进行解析，根据配置的规则来实现请求转发，实现方式有很多，比如Nginx, Contour, Haproxy等等
Ingress（以Nginx为例）的工作原理：
- 定义路由规则：用户通过 Kubernetes API 创建 Ingress 规则，指定域名与集群内服务的映射关系。
- 感知规则变化：Ingress 控制器（如基于 Nginx）实时监控 Kubernetes API，以便发现 Ingress 规则的更新。
- 生成配置：一旦检测到变化，Ingress 控制器自动生成相应的 Nginx 配置，以实现定义的路由规则。
- 更新 Nginx 配置：新生成的 Nginx 配置被应用到运行中的 Nginx 实例，无需重启服务即可动态更新路由规则。
- 流量转发：Nginx 作为反向代理，根据更新的配置，将外部请求转发到集群内正确的服务。
- SSL/TLS 终止（可选）：如果配置了 SSL/TLS，Nginx 还可以在转发前终止加密连接，提高安全性和效率。

Ingress安装部署

[root@k8s-master ~]# vi deploy.yaml
[root@k8s-master ~]# kubectl label node k8s-node1 node-role=ingress
node/k8s-node1 labeled
[root@k8s-master ~]# kubectl label node k8s-node2 node-role=ingress
node/k8s-node2 labeled
[root@k8s-master ~]# kubectl apply -f deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
daemonset.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
[root@k8s-master ~]# kubectl get pod 
No resources found in default namespace.
[root@k8s-master ~]# kubectl get pod  -n ingress-nginx
NAME                                   READY   STATUS              RESTARTS   AGE
ingress-nginx-admission-create-sgcg6   0/1     ContainerCreating   0          21s
ingress-nginx-admission-patch-2kdw2    0/1     CrashLoopBackOff    1          21s
ingress-nginx-controller-55776         0/1     ContainerCreating   0          21s
ingress-nginx-controller-vm965         0/1     ContainerCreating   0          21s
[root@k8s-master ~]# kubectl get pod  -n ingress-nginx -w
NAME                                   READY   STATUS              RESTARTS   AGE
ingress-nginx-admission-create-sgcg6   0/1     ContainerCreating   0          24s
ingress-nginx-admission-patch-2kdw2    0/1     CrashLoopBackOff    1          24s
ingress-nginx-controller-55776         0/1     ContainerCreating   0          24s
ingress-nginx-controller-vm965         0/1     ContainerCreating   0          24s
ingress-nginx-admission-create-sgcg6   0/1     Completed           0          25s
ingress-nginx-admission-create-sgcg6   0/1     Completed           0          25s
ingress-nginx-admission-patch-2kdw2    1/1     Running             2          28s
ingress-nginx-admission-patch-2kdw2    0/1     Completed           2          29s
ingress-nginx-admission-patch-2kdw2    0/1     Completed           2          29s
ingress-nginx-controller-55776         0/1     Running             0          87s
ingress-nginx-controller-vm965         0/1     Running             0          90s
^C[root@k8s-master ~]# kubectl get pod  -n ingress-nginx 
NAME                                   READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-sgcg6   0/1     Completed   0          95s
ingress-nginx-admission-patch-2kdw2    0/1     Completed   2          95s
ingress-nginx-controller-55776         0/1     Running     0          95s
ingress-nginx-controller-vm965         0/1     Running     0          95s
[root@k8s-master ~]# kubectl get pod  -n ingress-nginx -w
NAME                                   READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-sgcg6   0/1     Completed   0          105s
ingress-nginx-admission-patch-2kdw2    0/1     Completed   2          105s
ingress-nginx-controller-55776         1/1     Running     0          105s
ingress-nginx-controller-vm965         1/1     Running     0          105s

Ingress的HTTP代理

准备service和pod为了后面的实验比较方便，创建如下图所示的模型

[root@k8s-master ~]# vim tomcat-nginx.yaml 
[root@k8s-master ~]# kubectl create ns test 
namespace/test created
[root@k8s-master ~]# kubectl apply -f tomcat-nginx.yaml 
deployment.apps/tomcat-deployment created
service/tomcat-service created

[root@k8s-master ~]# kubectl get pod -n test  -w
NAME                                 READY   STATUS              RESTARTS   AGE
tomcat-deployment-7db86c59b7-7zbnc   0/1     ContainerCreating   0          50s
tomcat-deployment-7db86c59b7-r5xsn   0/1     ContainerCreating   0          50s
tomcat-deployment-7db86c59b7-sphwk   0/1     ImagePullBackOff    0          50s
tomcat-deployment-7db86c59b7-sphwk   0/1     ErrImagePull        0          70s
tomcat-deployment-7db86c59b7-sphwk   0/1     ImagePullBackOff    0          82s
tomcat-deployment-7db86c59b7-r5xsn   1/1     Running             0          4m29s
tomcat-deployment-7db86c59b7-7zbnc   1/1     Running             0          4m29s
tomcat-deployment-7db86c59b7-sphwk   1/1     Running             0          5m7s
^C[root@k8s-master ~]# kubectl get deploy,pod -n test 
NAME                                READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/tomcat-deployment   3/3     3            3           6m52s

NAME                                     READY   STATUS    RESTARTS   AGE
pod/tomcat-deployment-7db86c59b7-7zbnc   1/1     Running   0          6m52s
pod/tomcat-deployment-7db86c59b7-r5xsn   1/1     Running   0          6m52s
pod/tomcat-deployment-7db86c59b7-sphwk   1/1     Running   0          6m52s

Ingress配置

[root@k8s-master ~]# cat ingress-dep_lb.yaml 

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
  namespace: test
spec:
  ingressClassName: nginx
  rules:
  - host: www.test.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: svc-lb
            port:
              number: 80
  - host: tomcat.ctl.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: tomcat-service
            port:
              number: 80

[root@k8s-master ~]# kubectl apply -f ingress-dep_lb.yaml 
ingress.networking.k8s.io/nginx-ingress created
[root@k8s-master ~]# kubectl get service,ingress -n test
NAME                     TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
service/tomcat-service   LoadBalancer   10.96.166.18   <pending>     80:32593/TCP   10m

NAME                                      CLASS   HOSTS                         ADDRESS   PORTS   AGE
ingress.networking.k8s.io/nginx-ingress   nginx   www.test.com,tomcat.ctl.com             80      5s
[root@k8s-master ~]# kubectl get service,ingress -n test
NAME                     TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
service/tomcat-service   LoadBalancer   10.96.166.18   <pending>     80:32593/TCP   10m

NAME                                      CLASS   HOSTS                         ADDRESS                         PORTS   AGE
ingress.networking.k8s.io/nginx-ingress   nginx   www.test.com,tomcat.ctl.com   192.168.58.232,192.168.58.233   80      42s


[root@k8s-master ~]# kubectl get deploy,pod -n test 
NAME                                READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/tomcat-deployment   3/3     3            3           14m

NAME                                     READY   STATUS    RESTARTS   AGE
pod/tomcat-deployment-7db86c59b7-7zbnc   1/1     Running   0          14m
pod/tomcat-deployment-7db86c59b7-r5xsn   1/1     Running   0          14m
pod/tomcat-deployment-7db86c59b7-sphwk   1/1     Running   0          14m
[root@k8s-master ~]# kubectl get deploy,pod -n test  -o wide
NAME                                READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES                  SELECTOR
deployment.apps/tomcat-deployment   3/3     3            3           14m   tomcat       tomcat:8.5-jre10-slim   app=tomcat-pod

NAME                                     READY   STATUS    RESTARTS   AGE   IP               NODE        NOMINATED NODE   READINESS GATES
pod/tomcat-deployment-7db86c59b7-7zbnc   1/1     Running   0          14m   10.244.36.73     k8s-node1   <none>           <none>
pod/tomcat-deployment-7db86c59b7-r5xsn   1/1     Running   0          14m   10.244.36.72     k8s-node1   <none>           <none>
pod/tomcat-deployment-7db86c59b7-sphwk   1/1     Running   0          14m   10.244.169.131   k8s-node2   <none>           <none>
[root@k8s-master ~]# kubectl get service,ingress -n test
NAME                     TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
service/tomcat-service   LoadBalancer   10.96.166.18   <pending>     80:32593/TCP   14m

NAME                                      CLASS   HOSTS                         ADDRESS                         PORTS   AGE
ingress.networking.k8s.io/nginx-ingress   nginx   www.test.com,tomcat.ctl.com   192.168.58.232,192.168.58.233   80      4m34s
[root@k8s-master ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.58.231 k8s-master
192.168.58.232 k8s-node1
192.168.58.233 k8s-node2
192.168.58.232 www.test.com
192.168.58.233 tomcat.ctl.com

Ingress的HTTPS代理

创建证书和密钥

[root@k8s-master ~]# openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/C=CN/ST=BJ/L=BJ/O=nginx/CN=itopenlab.com"
Generating a 2048 bit RSA private key
.................................................................+++
.....+++
writing new private key to 'tls.key'
-----
[root@k8s-master ~]# kubectl create secret tls tls-secret --key tls.key --cert tls.crt
secret/tls-secret created

创建ingress-https.yaml


[root@k8s-master ~]#  vim ingress-https.yaml
[root@k8s-master ~]# kubectl apply -f ingress-https.yaml 
ingress.networking.k8s.io/ingress-https created
[root@k8s-master ~]# kubectl get ing ingress-https -n test
NAME            CLASS    HOSTS                          ADDRESS   PORTS     AGE
ingress-https   <none>   nginx.ctl.com,tomcat.ctl.com             80, 443   8s
[root@k8s-master ~]# kubectl describe ing ingress-https -n test
Name:             ingress-https
Namespace:        test
Address:          
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
  tls-secret terminates nginx.ctl.com,tomcat.ctl.com
Rules:
  Host            Path  Backends
  ----            ----  --------
  nginx.ctl.com   
                  /   nginx-service:80 (<error: endpoints "nginx-service" not found>)
  tomcat.ctl.com  
                  /   tomcat-service:8080 (10.244.169.131:8080,10.244.36.72:8080,10.244.36.73:8080)
Annotations:      <none>
Events:           <none>
[root@k8s-master ~]# cat ingress-https.yaml 



apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-https
  namespace: test
spec:
  tls:
  - hosts:
    - nginx.ctl.com
    - tomcat.ctl.com
    secretName: tls-secret # 指定秘钥
  rules:
  - host: nginx.ctl.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-service
            port:
              number: 80
  - host: tomcat.ctl.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: tomcat-service
            port:
              number: 8080
[root@k8s-master ~]# kubectl get ing ingress-https -n test -o wide
NAME            CLASS    HOSTS                          ADDRESS   PORTS     AGE
ingress-https   <none>   nginx.ctl.com,tomcat.ctl.com             80, 443   105s
[root@k8s-master ~]# kubectl get service,ingress -n test
NAME                     TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
service/tomcat-service   LoadBalancer   10.96.166.18   <pending>     80:32593/TCP   36m

NAME                                      CLASS    HOSTS                          ADDRESS                         PORTS     AGE
ingress.networking.k8s.io/ingress-https   <none>   nginx.ctl.com,tomcat.ctl.com                                   80, 443   2m1s
ingress.networking.k8s.io/nginx-ingress   nginx    www.test.com,tomcat.ctl.com    192.168.58.232,192.168.58.233   80        26m
[root@k8s-master ~]# curl https://nginx.ctl.com
^C
[root@k8s-master ~]# kubectl get service,ingress -n test
NAME                     TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
service/tomcat-service   LoadBalancer   10.96.166.18   <pending>     80:32593/TCP   37m

NAME                                      CLASS    HOSTS                          ADDRESS                         PORTS     AGE
ingress.networking.k8s.io/ingress-https   <none>   nginx.ctl.com,tomcat.ctl.com                                   80, 443   3m37s
ingress.networking.k8s.io/nginx-ingress   nginx    www.test.com,tomcat.ctl.com    192.168.58.232,192.168.58.233   80        27m