目标
这里假设,我们已经基本会使用k8s的kubectl命令进行部署了,也已经会自己打docker镜像推送到AWS ECR上面去了。而且,已经在云上准备好了AWS ECR镜像库和AWS EKS的k8s集群了。
这个前提上面,我们今天使用Helm Chart项目准备k8s的yaml配置文件来部署一个最简单的Springboot项目到AWS的k8s集群。
Helm项目结构
.
└── my-kubernetes-app
├── configmaps
├── crons
├── deployments
├── hpas
├── pdbs
├── podpriorities
├── pvcs
├── services
├── statefulsets
└── ...
上面这个是网上一位工友总结出来的Helm Chart目录结构。在这之前,还是先了解了解Helm是干什么的吧?Helm自身定位是给k8s的包管理器,但是,他的Helm Chart项目被我们现在这里用于管理生成k8s配置模板,来生成不同环境的k8s yaml配置文件。简单来说,Helm Chart就是一个根据模板生成配置文件的工具。
下面是我这次用的Helm目录结构:
./
├── Chart.yaml
├── templates
│ ├── configmaps
│ │ ├── demo.yaml
│ │ └── demo2.yaml
│ ├── ingress.yaml
│ ├── secrets
│ │ └── mysql.yaml
│ └── services
│ ├── demo.yaml
│ └── demo2.yaml
├── values-prod.yaml
├── values-uat.yaml
└── values.yaml
这里只有简单的secrets和services的简单文件夹划分。我这里只做java层面的拆分,数据库共用一个,嗯,我这里是属于伪微服务了。如果要进一步拆分的话,就像上面那位道友那样,把deployments从services里面统统拆出来。我这里是最简形态,比较原始的项目状态,后期,可以在这个基础上面加hpa。算了,精力有限,就先演化到这样吧!
这里我是先使用helm create命令,创建出一个超简单的helm项目的,具体命令如下:
helm create mychart
然后,在这个基础上面逐渐演化到上面那个secrets和services,ingress样子。
Chart.yaml
apiVersion: v2
name: my-demo
description: my-demo
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "1.16.0"
这里是Helm Chart基本信息。
values.yaml
ingress:
security:
groups: cloudfront-only
services:
demo:
name: demo
port: 8080
path: /api/demo
health:
path: /actuator/health
port: 8081
demo2:
name: demo2
port: 8080
path: /api/demo2
health:
path: /actuator/health
port: 8081
这里设置了ingress的安全组,这个cloudfront-only安全组,主要设置alb主要接收来自cloudfront的流量。这里业务服务的名称,端口,前缀路径,以及健康检查的路径和端口。
values-uat.yaml
replicas: 1
spring:
profiles:
active: uat
mysql:
config:
name: mysql
HOST: xxxx-uat.xxxxx.us-east-1.rds.amazonaws.com
DB_USERNAME: bXl1c2VybmFtZQ==
DB_PASSWORD: bXlwYXNzd29yZA==
docker:
services:
demo:
images: xxxxxxx.dkr.ecr.us-east-1.amazonaws.com/demo:latest
name: demo
demo2:
images: xxxxxxx.dkr.ecr.us-east-1.amazonaws.com/demo2:latest
name: demo2
这里主要是针对uat环境的设置,主要就是副本数,spring,mysql和docker镜像的配置。
templates/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ .Chart.Name }}-ingress
namespace: {{ .Release.Namespace }}
annotations:
alb.ingress.kubernetes.io/load-balancer-name: {{ .Chart.Name }}-ingress
alb.ingress.kubernetes.io/security-groups: {{ .Values.ingress.security.groups }}
alb.ingress.kubernetes.io/manage-backend-security-group-rules: "true"
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
spec:
ingressClassName: alb
rules:
- http:
paths:
- backend:
service:
name: {{ .Values.services.demo.name }}
port:
number: {{ .Values.services.demo.port }}
path: {{ .Values.services.demo.path }}
pathType: Prefix
这里主要就是k8s的ingress配置,主要就是alb配置和具体服务的网关配置。
templates/secrets/mysql.yaml
apiVersion: v1
data:
DB_USERNAME: {{ .Values.mysql.DB_USERNAME }}
DB_PASSWORD: {{ .Values.mysql.DB_PASSWORD }}
kind: Secret
metadata:
name: {{ .Values.mysql.config.name }}-secret
namespace: {{ .Release.Namespace }}
这里是mysql的secrets配置。
templates/services/demo.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: {{ .Values.services.demo.name }}
name: {{ .Values.services.demo.name }}
namespace: {{ .Release.Namespace }}
spec:
replicas: {{ .Values.replicas }}
selector:
matchLabels:
app: {{ .Values.services.demo.name }}
template:
metadata:
labels:
app: {{ .Values.services.demo.name }}
spec:
containers:
- image: {{ .Values.docker.services.demo.images }}
name: {{ .Values.docker.services.demo.name }}
envFrom:
- secretRef:
name: {{ .Values.mysql.config.name }}-secret
- configMapRef:
name: {{ .Values.services.demo.name }}-configmap
# resources:
# requests:
# memory: "2Gi"
# cpu: "2"
# limits:
# memory: "2Gi"
# cpu: "2"
# 准备检查,通过则接入流量
readinessProbe:
httpGet:
path: {{ .Values.services.demo.health.path }}
port: {{ .Values.services.demo.health.port }}
# 活力检查,不通过时重启容器
livenessProbe:
httpGet:
path: {{ .Values.services.demo.health.path }}
port: {{ .Values.services.demo.health.port }}
---
apiVersion: v1
kind: Service
metadata:
labels:
app: demo
name: {{ .Values.services.demo.name }}
namespace: {{ .Release.Namespace }}
annotations:
alb.ingress.kubernetes.io/healthcheck-path: {{ .Values.services.demo.health.path }}
alb.ingress.kubernetes.io/healthcheck-port: '{{ .Values.services.demo.health.port }}'
spec:
ports:
- name: http
port: {{ .Values.services.demo.port }}
targetPort: {{ .Values.services.demo.port }}
selector:
app: {{ .Values.services.demo.name }}
type: ClusterIP
这里主要配置了demo服务的Deployment和Service配置。
templates/services/demo2.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: {{ .Values.services.demo2.name }}
name: {{ .Values.services.demo2.name }}
namespace: {{ .Release.Namespace }}
spec:
replicas: {{ .Values.replicas }}
selector:
matchLabels:
app: {{ .Values.services.demo2.name }}
template:
metadata:
labels:
app: {{ .Values.services.demo2.name }}
spec:
containers:
- image: {{ .Values.docker.services.demo2.images }}
name: {{ .Values.docker.services.demo2.name }}
envFrom:
- secretRef:
name: {{ .Values.mysql.config.name }}-secret
- configMapRef:
name: {{ .Values.services.demo2.name }}-configmap
# resources:
# requests:
# memory: "2Gi"
# cpu: "2"
# limits:
# memory: "2Gi"
# cpu: "2"
# 准备检查,通过则接入流量
readinessProbe:
httpGet:
path: {{ .Values.services.demo2.health.path }}
port: {{ .Values.services.demo2.health.port }}
# 活力检查,不通过时重启容器
livenessProbe:
httpGet:
path: {{ .Values.services.demo2.health.path }}
port: {{ .Values.services.demo2.health.port }}
---
apiVersion: v1
kind: Service
metadata:
labels:
app: {{ .Values.services.demo2.name }}
name: {{ .Values.services.demo2.name }}
namespace: {{ .Release.Namespace }}
annotations:
alb.ingress.kubernetes.io/healthcheck-path: {{ .Values.services.demo2.health.path }}
alb.ingress.kubernetes.io/healthcheck-port: '{{ .Values.services.demo2.health.port }}'
spec:
ports:
- name: http
port: {{ .Values.services.demo2.port }}
targetPort: {{ .Values.services.demo2.port }}
selector:
app: {{ .Values.services.demo2.name }}
type: ClusterIP
configmaps/demo.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Values.services.demo.name }}-configmap
namespace: {{ .Release.Namespace }}
data:
SPRING_PROFILES_ACTIVE: {{ .Values.spring.profiles.active }}
MYSQL_HOST: {{ .Values.mysql.HOST }}
configmaps/demo2.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Values.services.demo2.name }}-configmap
namespace: {{ .Release.Namespace }}
data:
SPRING_PROFILES_ACTIVE: {{ .Values.spring.profiles.active }}
MYSQL_HOST: {{ .Values.mysql.HOST }}
Helm部署
helm install -f ./values-uat.yaml my-demo ./ -n my-demo --create-namespace
输出例子:
NAME: my-demo
LAST DEPLOYED: Thu Mar 14 15:19:53 2024
NAMESPACE: my-demo
STATUS: deployed
REVISION: 1
TEST SUITE: None
Helm检查部署
helm ls -n my-demo
结果如下:
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
my-demo my-demo 1 2024-03-14 15:19:53.658731 +0800 CST deployed my-demo-0.1.0 1.16.0
kubectl检查
# 检查ingress网关
kubectl get ingress -n my-demo
# 检查svc
kubectl get svc -n my-demo
# 检查deploy
kubectl get deploy -n my-demo
# 检查pods
kubectl get pods -n my-demo
页面检查
这里不涉及cloudfront与alb的配置,这里我们直接通过cloudfront检查我们部署的效果,具体效果如下:
总结
这里主要介绍Helm Chart项目来编写k8s配置文件的项目。哎!现在运维都有要写这么多代码了。
源代码:https://github.com/fxtxz2/helm-chart-demo
参考
- Developer’s Guide to Writing a Good Helm Chart