K8S暴露pod内多个端口
一、背景
公司统一用的某个底包跑jar服务，只暴露了8080端口
二、需求
由于有些服务在启动jar服务后，会启动多个端口，除了8080端口，还有别的端口需要暴露，我这里就还需要暴露9999端口。
注：解决办法其实是可以直接改底包就好了，在底包中多暴露几个端口，但是我这边因为无法改底包，所以只能通过下面的办法解决。
三、解决办法
我们平时在打版升级的过程中，会基于底包写dockerfile来替换最新的jar包得到最终的镜像，所以可以这个dockerfile中添加暴露9999端口，这样同样也可以增加容器端口暴露，如下
$ cat dockerfile
FROM 10.0.8.56/basis-images/basis:tomcat
ARG jar_name
RUN rm -rf /usr/local/tomcat/*
ADD ./target/${jar_name}.jar /usr/local/tomcat
ADD ./start.sh /usr/local/tomcat
EXPOSE 9999
RUN chmod +x /usr/local/tomcat/start.sh
$ docker build --build-arg jar_name=nsw-ai-video . -t 10.0.8.56/nsyai-test/nsw-ai-video:2023-07-28-15-40
$ docker push 10.0.8.56/nsyai-test/nsw-ai-video:2023-07-28-15-40四、实验在docker上跑容器，验证是否暴露出8080和9999端口（10.0.8.56是我的harbor私有镜像仓库）
[ yukw @ docker-work01 10.0.8.59 ] ~
$ docker login 10.0.8.56
Username: yukw
Password:
WARNING! Your password will be stored unencrypted in /home/yukw/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[ yukw @ docker-work01 10.0.8.59 ] ~
$ docker run -d -P --name nsyai-test 10.0.8.56/nsyai-test/nsw-ai-video:2023-07-28-15-40
Unable to find image '10.0.8.56/nsyai-test/nsw-ai-video:2023-07-28-15-40' locally
2023-07-28-15-40: Pulling from nsyai-test/nsw-ai-video
a8c7037c15e9: Pull complete
7f59206c4cb3: Pull complete
d6593d2ee432: Pull complete
47613084598b: Pull complete
7ef22be88035: Pull complete
edf70be6f818: Pull complete
af72e686cb89: Pull complete
376658e1b07e: Pull complete
6991c8295d7f: Pull complete
f0a023d2bec5: Pull complete
9356db0572c6: Pull complete
1cbc500b22f4: Pull complete
2a8383c1d611: Pull complete
962207b93da3: Pull complete
9fdef278ff07: Pull complete
8cc25cf21f3b: Pull complete
Digest: sha256:e07a648e671746f4408565b2237584303cfdfb7d5a451adfa707dda3fc87d670
Status: Downloaded newer image for 10.0.8.56/nsyai-test/nsw-ai-video:2023-07-28-15-40
e11553520d6c4d94c71d8d11a699bd4d1c6df8202d4e1ec15b28ca1bcd21ff25
[ yukw @ docker-work01 10.0.8.59 ] ~
$ docker ps -a |grep 'nsyai-test'
e11553520d6c 10.0.8.56/nsyai-test/nsw-ai-video:2023-07-28-15-40 "/usr/local/tomcat/s…" 8 seconds ago Up 7 seconds 0.0.0.0:49154->8080/tcp, 0.0.0.0:49153->9999/tcp nsyai-test
9fd678ee8eeb 10.0.8.56/nsyai-test/nsyai-web:2023-07-12-12-01 "/docker-entrypoint.…" 2 weeks ago Up 2 weeks 80/tcp my-nsyai-test
[ yukw @ docker-work01 10.0.8.59 ] ~
$ docker port e11553520d6c
8080/tcp -> 0.0.0.0:49154
9999/tcp -> 0.0.0.0:49153实验发现，端口暴露成功
容器8080端口随机映射到了宿主机49154端口
容器9999端口随机映射到了宿主机49153端口
五、编写dp.yaml
# cat dp.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "63"
description: video模块
labels:
k8s-app: nsw-ai-video
qcloud-app: nsw-ai-video
name: nsw-ai-video
namespace: nsyai-test
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: nsw-ai-video
qcloud-app: nsw-ai-video
template:
metadata:
labels:
k8s-app: nsw-ai-video
qcloud-app: nsw-ai-video
spec:
containers:
- name: nsw-ai-video
image: 10.0.8.56/nsyai-test/nsw-ai-video:2023-07-28-15-40
imagePullPolicy: Always
livenessProbe:
failureThreshold: 5
initialDelaySeconds: 180
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: 8080
timeoutSeconds: 6
readinessProbe:
failureThreshold: 5
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: 8080
timeoutSeconds: 60
resources:
limits:
cpu: 2000m
memory: 2Gi
requests:
cpu: "1"
memory: 512Mi
ports:
- containerPort: 8080
name: image-port
protocol: TCP
- containerPort: 9999
name: xxl-job-port
protocol: TCP
imagePullSecrets:
- name: nsw-harbor-secret
containerPort是在pod控制器中定义的、pod中的容器需要暴露的端口
六、编写svc.yaml
# cat svc.yaml
apiVersion: v1
kind: Service
metadata:
name: nsw-ai-video
namespace: nsyai-test
spec:
externalTrafficPolicy: Cluster
ports:
- name: 8080-8080-tcp
nodePort: 30083
port: 8080
protocol: TCP
targetPort: 8080
- name: 9999-9999-tcp
nodePort: 30084
port: 9999
protocol: TCP
targetPort: 9999
selector:
k8s-app: nsw-ai-video
qcloud-app: nsw-ai-video
type: NodePort
七、应用配置清单
# kubectl apply -f dp.yaml
# kubectl apply -f svc.yaml
# kubectl get svc -n nsyai-test
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nsw-ai-gateway ClusterIP 10.0.0.107 <none> 8080/TCP 16d
nsw-ai-video NodePort 10.0.0.119 <none> 8080:30083/TCP,9999:30084/TCP 3h25m
nsyai-pc-nginx NodePort 10.0.0.185 <none> 80:30082/TCP 16d
总结：
1、从上面可以发现，在制作pod镜像中EXPOSE暴露了两个端口，这个是容器本身需要暴露的端口，在dp.yaml中配置了两个containerPort，这个是pod中的容器需要暴露的端口，在svc.yaml中配置了nodePort，port，targetport，分别代表宿主机端口，service端口和容器端口。
好了，这就是K8S暴露pod内多个端口的办法了，如有问题可与博主一起交流讨论！
