tekton 发布 kubernetes 应用
基于Kubernetes 服务部署 Tekton Pipeline 实例，部署完成后使用tekton来完成源码拉取、应用打包、镜像推送和应用部署。
本文实现一个 golang-helloworld 项目 CI/CD 的完整流程，具体包括以下步骤：
- 从 gitee 仓库拉取代码，将源码构建成二进制文件
- 根据 Dockerfile 构建镜像并推送到阿里云ACR镜像仓库
- 使用sed命令替换yaml文件中的镜像地址为上一步构建的镜像
- 使用 kubectl apply -f 命令部署yaml文件到kubernetes集群
示例git仓库：https://gitee.com/willzhangee/tekton-golang-demo
创建serviceaccount
镜推送到外部镜像仓库需要进行认证，创建登录阿里云ACR仓库的secret
kubectl create secret docker-registry aliyun-acr \
--docker-server=https://registry.cn-shenzhen.aliyuncs.com \
--docker-username=<your-username> \
--docker-password=<your-password> \
--dry-run=client -o json | jq -r '.data.".dockerconfigjson"' | base64 -d > /tmp/config.json
kubectl create secret generic docker-config --from-file=/tmp/config.json
创建kubernetes secret
kubectl create secret generic kubernetes-config --from-file=/root/.kube/config
创建serviceAccount
$ cat serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: build-bot
secrets:
- name: docker-config
- name: kubernetes-config
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: tekton-kubectl-role
rules:
- apiGroups:
- "*"
resources:
- pods
- deployments
- deployments/scale
- deployments/status
verbs:
- get
- list
- watch
- create
- delete
- patch
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: tekton-kubectl-binding
subjects:
- kind: ServiceAccount
name: build-bot
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: tekton-kubectl-role
应用yaml文件
kubectl apply -f serviceaccount.yaml
创建 git-clone task
在执行镜像构建前Dockerfile存放在git仓库中，需要将代码克隆到本地，需要安装git-clone task，这里使用官方task。
kubectl apply -f \
https://raw.githubusercontent.com/tektoncd/catalog/main/task/git-clone/0.9/git-clone.yaml
创建kaniko-build task
创建kaniko-build task，用于构建dokcer镜像，基于官方kaniko-task改造。
$ cat kaniko-build-task.yaml
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: kaniko-build
spec:
params:
- name: IMAGE_URL
description: Name (reference) of the image to build.
- name: IMAGE_TAG
description: Tag to apply to the built image
default: latest
- name: DOCKERFILE
description: Path to the Dockerfile to build.
default: ./Dockerfile
- name: CONTEXT
description: The build context used by Kaniko.
default: ./
- name: EXTRA_ARGS
type: array
default: []
- name: BUILDER_IMAGE
description: The image on which builds will run (default is v1.5.1)
default: gcr.io/kaniko-project/executor:v1.5.1@sha256:c6166717f7fe0b7da44908c986137ecfeab21f31ec3992f6e128fff8a94be8a5
workspaces:
- name: source
description: Holds the context and Dockerfile
- name: dockerconfig
description: Includes a docker `config.json`
optional: true
mountPath: /kaniko/.docker
results:
- name: IMAGE_DIGEST
description: Digest of the image just built.
- name: IMAGE_URL
description: URL of the image just built.
steps:
- name: build-and-push
workingDir: $(workspaces.source.path)
image: $(params.BUILDER_IMAGE)
args:
- $(params.EXTRA_ARGS)
- --dockerfile=$(params.DOCKERFILE)
- --context=$(workspaces.source.path)/$(params.CONTEXT)
- --destination=$(params.IMAGE_URL):$(params.IMAGE_TAG)
- --digest-file=$(results.IMAGE_DIGEST.path)
securityContext:
runAsUser: 0
应用yaml文件
kubectl apply -f kaniko-build-task.yaml
创建kubernetes-deploy task
创建kubernetes-deploy task，用于部署yaml文件到kubernetes集群。
$ cat kubernetes-deploy-task.yaml
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: kubernetes-deploy
spec:
workspaces:
- name: manifest-dir
- name: kubeconfig-dir
mountPath: /root/.kube
params:
- name: pathToYamlFile
description: The path to the yaml file to deploy within the git source
default: deployment.yaml
- name: IMAGE_URL
- name: IMAGE_TAG
- name: KUBECTL_IMAGE
default: docker.io/bitnami/kubectl:latest
steps:
- name: run-kubectl
image: $(params.KUBECTL_IMAGE)
workingDir: $(workspaces.manifest-dir.path)
script: |
sed -i s#IMAGE#$(params.IMAGE_URL)#g $(params.pathToYamlFile)
sed -i s#TAG#$(params.IMAGE_TAG)#g $(params.pathToYamlFile)
kubectl apply -f $(params.pathToYamlFile)
应用yaml文件
kubectl apply -f kubernetes-deploy-task.yaml
创建pipeline和pipelinerun
$ cat pipeline-run.yaml
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: devops-hello-world-pipeline
spec:
workspaces:
- name: shared-data
- name: docker-config
- name: kubernetes-config
params:
# git-clone
- name: git_url
type: string
- name: revision
type: string
- name: gitInitImage
type: string
# kaniko-build
- name: dockerfile
type: string
description: reference of the image to build
- name: builder_image
type: string
description: reference of the image to build
- name: image_url
description: Url of image repository
- name: image_tag
description: Tag to apply to the built image
default: latest
# kubernetes-deploy
- name: kubectl_image
type: string
tasks:
- name: clone
taskRef:
name: git-clone
workspaces:
- name: output
workspace: shared-data
params:
- name: url
value: $(params.git_url)
- name: revision
value: $(params.revision)
- name: gitInitImage
value: $(params.gitInitImage)
- name: build-push-image
params:
- name: DOCKERFILE
value: $(params.dockerfile)
- name: IMAGE_URL
value: $(params.image_url)
- name: IMAGE_TAG
value: $(tasks.clone.results.commit)
- name: BUILDER_IMAGE
value: $(params.builder_image)
taskRef:
name: kaniko
runAfter:
- clone
workspaces:
- name: source
workspace: shared-data
- name: dockerconfig
workspace: docker-config
- name: deploy-to-k8s
taskRef:
name: kubernetes-deploy
params:
- name: KUBECTL_IMAGE
value: $(params.kubectl_image)
- name: IMAGE_URL
value: $(params.image_url)
- name: IMAGE_TAG
value: $(tasks.clone.results.commit)
- name: pathToYamlFile
value: deployment.yaml
workspaces:
- name: manifest-dir
workspace: shared-data
- name: kubeconfig-dir
workspace: kubernetes-config
runAfter:
- build-push-image
---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: devops-hello-world-pipeline-run-
spec:
serviceAccountName: build-bot
pipelineRef:
name: devops-hello-world-pipeline
params:
# git-clone
- name: git_url
value: https://gitee.com/willzhangee/tekton-golang-demo.git
- name: revision
value: master
- name: gitInitImage
#value: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:latest
value: dyrnq/tektoncd-pipeline-cmd-git-init:latest
# kaniko
- name: dockerfile
value: ./Dockerfile
- name: builder_image
# value: gcr.io/kaniko-project/executor:v1.5.1@sha256:c6166717f7fe0b7da44908c986137ecfeab21f31ec3992f6e128fff8a94be8a5
value: docker.io/bitnami/kaniko:latest
- name: image_url
value: registry.cn-shenzhen.aliyuncs.com/cnmirror/devops-hello-world
- name: image_tag
value: latest
# kubernetes-deploy
- name: kubectl_image
value: 'docker.io/bitnami/kubectl:latest'
workspaces:
- name: shared-data
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
storageClassName: openebs-hostpath
resources:
requests:
storage: 1Gi
- name: docker-config
secret:
secretName: docker-config
- name: kubernetes-config
secret:
secretName: kubernetes-config
参数说明：
- gitInitImage：执行git clone任务的镜像，官方镜像无法访问，推荐在docekrhub中查找替代镜像
- builder_image：执行kaniko 构建任务的镜像，官方镜像无法访问，推荐在docekrhub中查找替代镜像
- image_url：最终构建的应用镜像
- serviceAccountName：指定
serviceAccountName用于认证 - shared-data workspace：用于在不同任务之间共享数据，PipelineRun中定义了
volumeClaimTemplate类型的workspaces，能够动态申请所需的持久卷，使用kubectl get storageclass命令，确认k8s集群有默认可用的storageclass资源可用，本示例输出为openebs-hostpath (default) - docker-config workspace：用于镜像仓库认证的secret卷，将secret中的
config.json挂载到/kaniko/.docker下 - kubernetes-config：用于访问kubernetes，挂载到/root/.kube目录下
应用yaml文件
kubectl create -f pipeline-run.yaml
查看pipelinerun执行结果
连接到kubernetes 确认部署的应用
root@kube001:~# kubectl get pods -l run=go-web-app
NAME READY STATUS RESTARTS AGE
go-web-app-79454cfdd7-dcz7p 1/1 Running 0 64s
查看镜像信息
root@kube001:~# kubectl get pods go-web-app-79454cfdd7-dcz7p -o jsonpath='{.spec.containers[0].image}'
registry.cn-shenzhen.aliyuncs.com/cnmirror/devops-hello-world:927ec5cc665690ad798ffbbd02a8db520692951e
参考：https://juejin.cn/post/7073347226772340749
![[SWPUCTF 2021 新生赛]error](https://img-blog.csdnimg.cn/direct/b839b5328b0c40a0b3fef850e8bbcd14.png#pic_center)
