K8s: Ingress对象, 创建Ingress控制器, 创建Ingress资源并暴露服务

Ingress对象

1 ）概述

Ingress 是对集群中服务的外部访问进行管理的 API 对象，典型的访问方式是 HTTP
Ingress-nginx 本质是网关，当你请求 abc.com/service/a, Ingress 就把对应的地址转发给你，底层运行了一个 nginx
但 K8s 为什么不直接使用 nginx 呢，是因为 K8s 也需要把转发的路由规则纳入它的配置管理
变成 ingress 对象，所有才有 ingress 这个资源对象, Ingress 公开了从集群外部到集群内服务的 HTTP 和 HTTPS 路由
流量路由由 Ingress 资源上定义的规则控制
所以，它的功能类似 Nginx，可以根据域名、路径把请求转发到不同的 Service
Ingress 为外部访问集群提供了一个统一入口，避免了对外暴露集群端口，也可以配置 https

2 ）示例图

下面是一个将所有流量都发送到同一 Service 的简单 Ingress 示例

在 Service 层已经可以对外提供服务了，但是
在后端 Service 安全权限非常高的情况下，直连 Service 层风险非常大
从客户端里，通过Ingress的controller调度到Ingress服务，Ingress 可以理解为一个反向代理服务
这样，避免了直连Service层的风险，所以，Ingress 也类似于网关层，调度到Service之后
再由底层调度到相关的 Pod 中访问对应的服务
Ingress 有两种实践方法
- 一种是, Ingress Nginx 实现，在Nginx官方中有相关说明
- 另一种就是在 K8s 中的实践
对于典型生产环境来说，有上图这样一套调用链
可以将 Ingress 配置为服务提供外部可访问的 URL、负载均衡流量、终止 SSL/TLS，以及提供基于名称的虚拟主机等能力
Ingress控制器通常负责通过负载均衡器来实现 Ingress

3 ）最小 Ingress 资源示例

定义 ing-min.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minimal-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:   # 除了 http 还可以定义其他路由规则
      paths:  # 这个名称意味着可以定义多个 path
        - path: /testpath
          pathType: Prefix
          backend:
            service:
              name: test
              port:
                number: 80

基于以上的配置定义，客户端可以通过比如 xxx.com/testpath 请求
通过这个请求，会被 Ingress 捕获，根据这个请求规则，会匹配后端的 backend service
这个 service 名称就是 k8s 中的 service 名称，下面是对应的端口号
通过这个转发，类似于 nginx，实现路由规则的http转发
关于 Ingress 规则，每个 HTTP 规则都包含以下信息
- 1 ）可选的 host
  - 在此示例中，未指定 host，因此该规则适用于通过指定 IP 地址的所有入站 HTTP 通信
  - 如果提供了 host（例如 foo.bar.com），则 rules 适用于该 host
- 2 ）路径列表 paths（例如，/testpath）
  - 每个路径都有一个由 serviceName 和 servicePort 定义的关联后端
  - 在负载均衡器将流量定向到引用的服务之前，主机和路径都必须匹配传入请求的内容
- 3 ）backend（后端）
  - 是 Service 文档中所述的服务和端口名称的组合
  - 与规则的 host 和 path 匹配的对 Ingress 的 HTTP（和 HTTPS ）请求将发送到列出的 backend

4 ）Ingress 控制器

关于 Ingress 控制器
- 为了让 Ingress 资源工作，集群必须有一个正在运行的 Ingress 控制器
- 与其他类型的控制器不同，Ingress 控制器不是随集群自动启动的
版本对应
- 介于之前试错的经验，在各个版本的K8s上部署不同的yaml配置，会导致各种不一样的报错，
- 我在官方github上找到这个对应的版本信息，如下
  - https://github.com/kubernetes/ingress-nginx
  - 目前我的K8s的版本是1.22.4，所以这个控制器最高可以选择版本 v1.4.0
  - https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.4.0/deploy/static/provider/cloud/deploy.yaml
  - 这个文件下载下来后，需要做一些修改
  - 注意：如果上述github无法访问，可以找gitee中对应的镜像里的对应的版本

安装 Ingress 控制器

这里创建一个 ing-nginx-ctrl.yaml 文件

和上面官方不同的几点是:

在第一个Service中找到 spec 下
- externalTrafficPolicy: Local 修改为 externalTrafficPolicy: Cluster
- 并在这个配置的上面添加一行: clusterIP: 10.1.211.240
- 在 name: http 下添加一行 nodePort: 31686
- 在 name: https 下添加一行 ``
- 找到 type: LoadBalancer 修改为 type: NodePort
替换通用镜像
- 先找到 image: registry.k8s.io/ingress-nginx/controller:v1.4.0@sha256:34ee929b111ffc7aa426ffd409af44da48e5a0eea1eb2207994d9e0c0882d143
- 修改为: image: registry.aliyuncs.com/google_containers/nginx-ingress-controller:v1.4.0
- 再找到 image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20220916-gd32f8c343@sha256:39c5b2e3310dc4264d638ad28d9d1d96c4cbb2b2dcfb52368fe4e3c63f61e10f
- 修改为: image: registry.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.0
- 注意，这些镜像可以先拉到本地
  - $ sudo docker pull registry.aliyuncs.com/google_containers/nginx-ingress-controller:v1.4.0
  - $ sudo docker pull registry.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.0

修改后的 ing-nginx-ctrl.yaml 文件内容如下

apiVersion: v1
kind: Namespace
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx
  namespace: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - configmaps
  - pods
  - secrets
  - endpoints
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - networking.k8s.io
  resources:
  - ingressclasses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resourceNames:
  - ingress-controller-leader
  resources:
  - configmaps
  verbs:
  - get
  - update
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - create
- apiGroups:
  - coordination.k8s.io
  resourceNames:
  - ingress-controller-leader
  resources:
  - leases
  verbs:
  - get
  - update
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - create
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - discovery.k8s.io
  resources:
  - endpointslices
  verbs:
  - list
  - watch
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - endpoints
  - nodes
  - pods
  - secrets
  - namespaces
  verbs:
  - list
  - watch
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - networking.k8s.io
  resources:
  - ingressclasses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - discovery.k8s.io
  resources:
  - endpointslices
  verbs:
  - list
  - watch
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx-admission
rules:
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - validatingwebhookconfigurations
  verbs:
  - get
  - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx-admission
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: v1
data:
  allow-snippet-annotations: "true"
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  clusterIP: 10.1.211.240
  externalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - appProtocol: http
    name: http
    nodePort: 31686
    port: 80
    protocol: TCP
    targetPort: http
  - appProtocol: https
    name: https
    nodePort: 30036
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: NodePort
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx-controller-admission
  namespace: ingress-nginx
spec:
  ports:
  - appProtocol: https
    name: https-webhook
    port: 443
    targetPort: webhook
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  minReadySeconds: 0
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/name: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
    spec:
      containers:
      - args:
        - /nginx-ingress-controller
        - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
        - --election-id=ingress-controller-leader
        - --controller-class=k8s.io/ingress-nginx
        - --ingress-class=nginx
        - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
        - --validating-webhook=:8443
        - --validating-webhook-certificate=/usr/local/certificates/cert
        - --validating-webhook-key=/usr/local/certificates/key
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: LD_PRELOAD
          value: /usr/local/lib/libmimalloc.so
        image: registry.aliyuncs.com/google_containers/nginx-ingress-controller:v1.4.0
        imagePullPolicy: IfNotPresent
        lifecycle:
          preStop:
            exec:
              command:
              - /wait-shutdown
        livenessProbe:
          failureThreshold: 5
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: controller
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        - containerPort: 443
          name: https
          protocol: TCP
        - containerPort: 8443
          name: webhook
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          requests:
            cpu: 100m
            memory: 90Mi
        securityContext:
          allowPrivilegeEscalation: true
          capabilities:
            add:
            - NET_BIND_SERVICE
            drop:
            - ALL
          runAsUser: 101
        volumeMounts:
        - mountPath: /usr/local/certificates/
          name: webhook-cert
          readOnly: true
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      volumes:
      - name: webhook-cert
        secret:
          secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx-admission-create
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.4.0
      name: ingress-nginx-admission-create
    spec:
      containers:
      - args:
        - create
        - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
        - --namespace=$(POD_NAMESPACE)
        - --secret-name=ingress-nginx-admission
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: registry.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.0
        imagePullPolicy: IfNotPresent
        name: create
        securityContext:
          allowPrivilegeEscalation: false
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: OnFailure
      securityContext:
        fsGroup: 2000
        runAsNonRoot: true
        runAsUser: 2000
      serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx-admission-patch
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.4.0
      name: ingress-nginx-admission-patch
    spec:
      containers:
      - args:
        - patch
        - --webhook-name=ingress-nginx-admission
        - --namespace=$(POD_NAMESPACE)
        - --patch-mutating=false
        - --secret-name=ingress-nginx-admission
        - --patch-failure-policy=Fail
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: registry.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.0
        imagePullPolicy: IfNotPresent
        name: patch
        securityContext:
          allowPrivilegeEscalation: false
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: OnFailure
      securityContext:
        fsGroup: 2000
        runAsNonRoot: true
        runAsUser: 2000
      serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: nginx
spec:
  controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.4.0
  name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
  - v1
  clientConfig:
    service:
      name: ingress-nginx-controller-admission
      namespace: ingress-nginx
      path: /networking/v1/ingresses
  failurePolicy: Fail
  matchPolicy: Equivalent
  name: validate.nginx.ingress.kubernetes.io
  rules:
  - apiGroups:
    - networking.k8s.io
    apiVersions:
    - v1
    operations:
    - CREATE
    - UPDATE
    resources:
    - ingresses
  sideEffects: None

简单来说 ingress controller 实际在系统里面创建一系列的pod
本质上就是运行在 K8s服务器上的一系列的 pod, 通过 pod 来接管
外部到 K8s work node 上的请求，所以，它就是类似于 nginx 的组件

$ kubectl apply -f ing-nginx-ctrl.yaml

namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created

$ kubectl get all -n ingress-nginx 查看命名空间下的所有信息

  NAME                                            READY   STATUS      RESTARTS   AGE
pod/ingress-nginx-admission-create--1-8nbrv     0/1     Completed   0          65s
pod/ingress-nginx-admission-patch--1-2q9x9      0/1     Completed   3          65s
pod/ingress-nginx-controller-6747799754-v2vhq   1/1     Running     0          65s

NAME                                         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
service/ingress-nginx-controller             NodePort    10.1.211.240   <none>        80:31686/TCP,443:30036/TCP   65s
service/ingress-nginx-controller-admission   ClusterIP   10.1.195.73    <none>        443/TCP                      65s

NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/ingress-nginx-controller   1/1     1            1           65s

NAME                                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/ingress-nginx-controller-6747799754   1         1         1       65s

NAME                                       COMPLETIONS   DURATION   AGE
job.batch/ingress-nginx-admission-create   1/1           21s        65s
job.batch/ingress-nginx-admission-patch    1/1           44s        65s

这里，发现namespace为ingress-nginx的三个pod已经成功完成
status为Completed的两个pod为job类型资源，Completed表示job已经成功执行
status为Running的pod就是控制器

有了这样的一个组件在K8s平台运行起来之后，可以检查部署版本，粘贴如下
- $ POD_NAMESPACE=ingress-nginx
- $ POD_NAME=$(kubectl get pods -n $POD_NAMESPACE -l app.kubernetes.io/name=ingress-nginx --field-selector=status.phase=Running -o jsonpath='{.items[0].metadata.name}')
- $ kubectl exec -it $POD_NAME -n $POD_NAMESPACE -- /nginx-ingress-controller --version
```
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v1.4.0
  Build:         50be2bf95fd1ef480420e2aa1d6c5c7c138c95ea
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.19.10

-------------------------------------------------------------------------------
```

$ kubectl get svc -n ingress-nginx 查看可用Services

NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.1.211.240   <none>        80:31686/TCP,443:30036/TCP   9m49s
ingress-nginx-controller-admission   ClusterIP   10.1.195.73    <none>        443/TCP                      9m49s

到现在为止，服务已经搭建起来了，我们来验证一下
- $ curl node1.k8s:31686 或 curl node2.k8s:31686
- 说明: node1.k8s 或 node2.k8s 是可用的work node, 本地配置了 hosts，才可这样访问
- 如果结果显示如下，则表示服务已经通了
```
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
```
综上，ingress 的控制器已经搭建完毕

5 ）基于 ingress 控制器创建 ingress 资源，并对外暴露服务

在创建 ingress 资源之前，先部署我们的后端应用服务，这里做最简单的示例
- $ kubectl create deployment web --image=registry.cn-beijing.aliyuncs.com/qingfeng666/hello-app:1.0 基于 development 维护一个pod
```
deployment.apps/web created
```
- $ kubectl get po -w 监控pod的状态，等待 Running
```
NAME                   READY   STATUS    RESTARTS   AGE
web-6db77f5fdb-qkk6n   1/1     Running   0          7s
```
- $ kubectl expose deployment web --type=NodePort --port=8080 将 development 服务暴露出来
```
service/web exposed
```
- $ kubectl get svc 获取目前的服务
```
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)          AGE
kubernetes   ClusterIP   10.1.0.1     <none>        443/TCP          5d8h
web          NodePort    10.1.47.34   <none>        8080:32041/TCP   8s
```
- $ curl node1.k8s:32041 或 curl node2.k8s:32041
```
 Hello, world!
 Version: 1.0.0
 Hostname: web-6db77f5fdb-65wfv
```
  - 可见，在集群内部，我们的服务已经启动起来了
- 现在内部pod和Service已经就绪，现在可以进行创建 ingress 资源了
- $ vi ing-demo.yaml
```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-nginx
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
   ingressClassName: nginx
   rules:
   - host: hello-world.info
     http:
       paths:
       - path: /
         pathType: Prefix
         backend:
           service:
             name: web
             port:
               number: 8080
```
- $ kubectl apply -f ing-demo.yaml 创建 ingress 资源
```
ingress.networking.k8s.io/ingress-nginx created
```
- $ kubectl get ing 查看 ingress 资源
```
NAME            CLASS   HOSTS              ADDRESS        PORTS   AGE
ingress-nginx   nginx   hello-world.info   10.1.211.240   80      2m13s
```
- $ sudo vi /etc/hosts 添加一行, 对当前ip进行域名的配置
```
10.1.211.240  hello-world.info
```
- $ curl hello-world.info 访问域名，发现通了
```
Hello, world!
Version: 1.0.0
Hostname: web-6db77f5fdb-65wfv
```
- 这样，就完成了集群外的暴露，但是还需要再客户端机器或云服务器的域名解析，这里选择前者
  - 比如，在我的Mac电脑上连接当前 hello-world服务，这里前提是: Mac电脑和Centos可以连通
  - 在 Mac 上配置某个 Centos 的work node的host, $ sudo vi /etc/hosts
```
10.211.55.11  hello-world.info
```
  - 这里的 10.211.55.11 对应 work node 的 ip
- 在我的 Mac 上浏览器访问: http://hello-world.info:31686，如下
  - 像是这种访问不方便: http://hello-world.info:31686 这个端口比较麻烦
  - 可以修改成 80端口, 这样，就可以这样访问了：http://hello-world.info, 这里不演示了，参考如下
  - 参考: https://blog.csdn.net/qq_32060101/article/details/135691179
    - k8s修改NodePort支持80端口
  - 参考: https://blog.csdn.net/qq_32060101/article/details/135691441
    - ingress控制器修改NodePort成80端口