Gitlab: PHP项目CI/CD实践

1 说明

2 CI/CD

2.1 部署方式一：增量部署

2.1.1 目标服务器准备

2.2.2 Gitlab及Envoy脚本

2.2 部署方式二：镜像构建与部署

2.2.1 推送到私有化容器仓库

准备工作

脚本

要点

2.2.2 推送到hub.docker.com

准备工作

脚本

3 参考：

1 说明

以一个laravel blog项目为例，做dev分支的CI/CD实践
结合laravel envoy工具做多个远程服务器部署，分两种方式：A. 增量部署 B.镜像构建与部署

服务器：

Site	Server	IP	站点目录
team1-prj2.dev.ia	host001.dev.ia	192.168.0.130	/www/wwwroot/team1-prj2.dev.ia
team1-prj2.dev.ia	host002.dev.ia	192.168.0.131	/www/wwwroot/team1-prj2.dev.ia

2 CI/CD

2.1 部署方式一：增量部署

通过Lavavel/Envoy和git拉取新版本文件进行部署

2.1.1 目标服务器准备

php8.2, 安装所需扩展，务必在cli下测试是否正常, 有些被disable的functions要打开
composer self-update，兼容php8.2
使用Laravel/Envoy分发部署，确保Envoy.blade.php是utf8格式文件

站点目录结构

root@host001:/www/wwwroot/team1-prj2.dev.ia# tree -d -L 3 ./
./
├── current -> /www/wwwroot/team1-prj2.dev.ia/releases/default
├── releases
│   └── default
│       ├── app
│       ├── bootstrap
│       ├── config
│       ├── database
│       ├── public
│       ├── resources
│       ├── routes
│       ├── storage -> /www/wwwroot/team1-prj2.dev.ia/storage
│       ├── tests
│       └── vendor
└── storage
    ├── app
    │   └── public
    ├── framework
    │   ├── cache
    │   ├── sessions
    │   ├── testing
    │   └── views
    └── logs

说明：

team1-prj2.dev.ia	应用目录
team1-prj2.dev.ia/releases	版本发布目录，这里只设置了一个default目录，也可根据需要做日期变量发布
team1-prj2.dev.ia/current	链接到最新版本，被nginx访问的站点目录路径 current -> /www/wwwroot/team1-prj2.dev.ia/releases/default/
team1-prj2.dev.ia/storage	链接到最新版本的应用数据保存目录，如：日志，缓存等 storage -> /www/wwwroot/team1-prj2.dev.ia/storage
team1-prj2.dev.ia/.dev	.dev文件是运维人员建立的服务器定制环境文件，不进入仓库，链接到项目同名文件 .env -> /www/wwwroot/team1-prj2.dev.ia/.env*
team1-prj2.dev.ia/releases/default/.gitlab-ci.yml	gitlab 部署脚本
team1-prj2.dev.ia/releases/default/Envoy.blade.php	envoy 部署脚本

nginx配置

server
{
    listen 80;
    server_name team1-prj2.dev.ia;
    index index.php index.html index.htm default.php default.htm default.html;
    root /www/wwwroot/team1-prj2.dev.ia/current/public;
    #CERT-APPLY-CHECK--START
    # 用于SSL证书申请时的文件验证相关配置 -- 请勿删除
    include /www/server/panel/vhost/nginx/well-known/team1-prj2.dev.ia.conf;
    #CERT-APPLY-CHECK--END

    #SSL-START SSL相关配置，请勿删除或修改下一行带注释的404规则
    #error_page 404/404.html;
    #SSL-END

    #ERROR-PAGE-START  错误页配置，可以注释、删除或修改
    #error_page 404 /404.html;
    #error_page 502 /502.html;
    #ERROR-PAGE-END

    #PHP-INFO-START  PHP引用配置，可以注释或修改
    include enable-php-82.conf;
    #PHP-INFO-END

    #REWRITE-START URL重写规则引用,修改后将导致面板设置的伪静态规则失效
    include /www/server/panel/vhost/rewrite/team1-prj2.dev.ia.conf;
    #REWRITE-END

    #禁止访问的文件或目录
    location ~ ^/(\.user.ini|\.htaccess|\.git|\.env|\.svn|\.project|LICENSE|README.md)
    {
        return 404;
    }

    #一键申请SSL证书验证目录相关设置
    location ~ \.well-known{
        allow all;
    }

    #禁止在证书验证目录放入敏感文件
    if ( $uri ~ "^/\.well-known/.*\.(php|jsp|py|js|css|lua|ts|go|zip|tar\.gz|rar|7z|sql|bak)$" ) {
        return 403;
    }

    location ~ .*\.(gif|jpg|jpeg|png|bmp|swf)$
    {
        expires      30d;
        error_log /dev/null;
        access_log /dev/null;
    }

    location ~ .*\.(js|css)?$
    {
        expires      12h;
        error_log /dev/null;
        access_log /dev/null;
    }
    
    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }
    
    access_log  /www/wwwlogs/team1-prj2.dev.ia.log;
    error_log  /www/wwwlogs/team1-prj2.dev.ia.error.log;
}

2.2.2 Gitlab及Envoy脚本

.gitlab-ci.yml

# default:
#  image: edbizarro/gitlab-ci-pipeline-php:7.4
#default:
# image: bennybi/php8.2
# image: bennybi/php7.4

stages:
  - test
  - deploy

.init_ssh: &init_ssh |
  which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )
  eval $(ssh-agent -s)
  echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
  mkdir -p ~/.ssh
  chmod 700 ~/.ssh
  [[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config

cache:
  key: ${CI_COMMIT_REF_SLUG}
  paths:
    - vendor/

unit_test:
  stage: test
  tags:
    - php
  script:
    - cp .env.test .env
    - composer install
    - composer global require "laravel/envoy"
    - php artisan key:generate
    - php artisan migrate
    - vendor/bin/phpunit

deploy_dev:
  stage: deploy
  tags:
    - php
  environment:
    name: dev
    url: http://team1-prj2.dev.ia
  script:
    - *init_ssh
    - vendor/bin/envoy run deploy --branch="$CI_COMMIT_BRANCH" --commit="$CI_COMMIT_SHA"
  rules:
    - if: $CI_COMMIT_BRANCH == "dev"

deploy_live:
  stage: deploy
  tags:
    - php
  script:
    - *init_ssh
    - vendor/bin/envoy run deploy --commit="$CI_COMMIT_SHA"
  environment:
    name: live
    url: http://team1-prj2.dev.ia
  when: manual
  rules:
    - if: $CI_COMMIT_BRANCH == "live"

Envoy.blade.php

@servers(['local' => 'deployer@host001.dev.ia','staging' => 'deployer@host002.dev.ia'])

@setup
    $repository = 'git@host001.dev.ia:dev1/team1-prj2.git';
    $releases_dir = '/www/wwwroot/team1-prj2.dev.ia/releases';
    $app_dir = '/www/wwwroot/team1-prj2.dev.ia';
    $release = 'default';
    $new_release_dir = $releases_dir .'/'. $release;
    $user = get_current_user();
@endsetup

@story('deploy', ['on' => ['local','staging']])
    sync_repository
    run_composer
    update_symlinks
@endstory

@task('sync_repository')
    echo "Current User: {{$user}}, branch:{{$branch}}, commit:{{$commit}}"
    if [ -d "{{$new_release_dir}}" ]; then
        echo 'Pulling repository'
        cd {{ $new_release_dir }}
        git checkout {{ $branch }}
        git fetch
        git reset --hard HEAD
        git merge origin/{{ $branch }}
    else
        echo 'Cloning repository'
        [ -d {{ $releases_dir }} ] || mkdir {{ $releases_dir }}
        git clone --branch {{ $branch }}  --single-branch --depth 1 {{ $repository }} {{ $new_release_dir }}
        cd {{ $new_release_dir }}
        git reset --hard {{ $commit }}
        git config --global --add safe.directory '*'
    fi
@endtask

@task('run_composer')
    echo "Starting deployment ({{ $release }})"
    cd {{ $new_release_dir }}
    composer install --prefer-dist --no-scripts -q -o
@endtask

@task('update_symlinks')
    if [ ! -d "{{ $app_dir }}/current" ]; then
        echo "Linking storage directory"
        {{-- rm -rf {{ $new_release_dir }}/storage  --}}
        mv {{ $new_release_dir }}/storage {{ $app_dir }}
        ln -nfs {{ $app_dir }}/storage {{ $new_release_dir }}/storage
        
        echo 'Linking .env file'
        ln -nfs {{ $app_dir }}/.env {{ $new_release_dir }}/.env
        
        echo 'Linking current release'
        ln -nfs {{ $new_release_dir }} {{ $app_dir }}/current
        
        chmod 775 -Rf {{ $releases_dir }}
    fi
    chmod 775 -Rf {{ $new_release_dir }}/storage
    {{-- chown deployer:www -Rf {{ $new_release_dir }}/.git  --}}
@endtask

2.2 部署方式二：镜像构建与部署

2.2.1 推送到私有化容器仓库

准备工作

- 新建项目team1-prj1，初始化git 仓库为：http://host001.dev.ia:18181/dev1/team1-prj1.git

- 建好容器仓库（见前文相关部分）

- 需要在Admin Area->CI/CD->Variables添加docker访问用户变量

$LOCAL_REGISTRY_LOGIN = your docker username
$LOCAL_REGISTRY_PASSWORD

脚本

项目中添加Dockerfile 文件，这里用到的原型镜像是我之前定制的php8.2

# Set the base image for subsequent instructions
FROM bennybi/php8.2:latest

# Update packages
RUN apt-get update

# Install PHP and composer dependencies
# RUN apt-get install -qq git curl libmcrypt-dev libjpeg-dev libpng-dev libfreetype6-dev libbz2-dev

# Clear out the local repository of retrieved package files
RUN apt-get clean

# Install needed extensions
# Here you can install any other extension that you need during the test and deployment process
# RUN docker-php-ext-install mcrypt pdo_mysql zip

# Install Composer
RUN curl --silent --show-error "https://getcomposer.org/installer" | php -- --install-dir=/usr/local/bin --filename=composer

# Install Laravel Envoy
RUN composer global require "laravel/envoy=~1.0"

# Laravel? Add your user
RUN groupadd -g 1000 www
RUN useradd -u 1000 -ms /bin/bash -g www www

# Copy existing application directory contents
COPY . /var/www

# Copy existing application directory permissions
COPY --chown=www:www . /var/www

# Change current user to www
USER www

# Expose port 9000 and start php-fpm server
EXPOSE 9000
CMD ["php-fpm"]

.gitlab-ci.yml

default:
  image: docker:19.03.8
  services:
    - mysql:5.7
  before_script:
    - docker info

variables:
  MYSQL_DATABASE: test
  MYSQL_ROOT_PASSWORD: fa843c707ce26702
  DB_HOST: host001.dev.ia
  DB_USERNAME: developer

stages:
  - test
  - build
  # - deploy

unit_test:
  stage: test
  tags:
    - php
  script:
    - cp .env.test .env
    - composer install
    - php artisan key:generate
    - php artisan migrate
    - vendor/bin/phpunit

build_image:
  stage: build
  variables:
    REGISTRY_URL: 'http://host001.dev.ia:5050'
    DOCKER_IMAGE_TAG: 'host001.dev.ia:5050/dev1/team1-prj1'
  tags:
    - php
  script:
    - docker login ${REGISTRY_URL} -u $LOCAL_REGISTRY_LOGIN -p $LOCAL_REGISTRY_PASSWORD
    - docker build --output type=registry,oci-mediatypes=false --cache-from "${DOCKER_IMAGE_TAG}" -t "${DOCKER_IMAGE_TAG}" --push --provenance=false .
    # - docker push ${DOCKER_IMAGE_TAG}:latest
    - docker push ${DOCKER_IMAGE_TAG}
  rules:
    - if: $CI_COMMIT_BRANCH == "dev"

# deploy_dev:
#   stage: deploy
#   tags:
#     - php
#   script:
#     - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
#     - eval $(ssh-agent -s)
#     - ssh-add <(echo "$SSH_PRIVATE_KEY")
#     - mkdir -p ~/.ssh
#     - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
#     # - ~/.composer/vendor/bin/envoy run deploy --commit="$CI_COMMIT_SHA"
#   environment:
#     name: dev
#     url: http://team1-prj1.dev.ia
#   when: manual
#   rules:
#     - if: $CI_COMMIT_BRANCH == "dev"

要点

- 注意build时所需的参数，缺少会诱发错误： “Invalid tag: missing manifest digest”

We used the buildx flag “–output type=registry,oci-mediatypes=false” to generate a Docker v2.2 manifest list. We could set the value to “true” and generate an OCI manifest index, but the GitLab UI will incorrectly display “Invalid tag: missing manifest digest”.

推送结果图示：

2.2.2 推送到hub.docker.com

准备工作

- 新建项目team1-prj1，初始化git 仓库为：http://host001.dev.ia:18181/dev1/team1-prj1.git

- 构建的镜像将push到hub.docker.com，因此需要在Admin Area->CI/CD->Variables添加docker访问用户变量

$DOCKER_LOGIN = your docker username
$DOCKER_PASSWORD

脚本

Dockerfile

同上 ...

.gitlab-ci.yml

default:
  image: docker:19.03.8
  services:
    - mysql:5.7
  before_script:
    - docker info

variables:
  MYSQL_DATABASE: test
  MYSQL_ROOT_PASSWORD: fa843c707ce26702
  DB_HOST: host001.dev.ia
  DB_USERNAME: developer

stages:
  - test
  - build
  # - deploy

unit_test:
  stage: test
  tags:
    - php
  script:
    - cp .env.test .env
    - composer install
    - php artisan key:generate
    - php artisan migrate
    - vendor/bin/phpunit

build_image:
  stage: build
  variables:
    DOCKER_IMAGE_TAG: 'bennybi/team1-prj1'
  tags:
    - php
  script:
    - docker build --cache-from "${DOCKER_IMAGE_TAG}" -t "${DOCKER_IMAGE_TAG}" .
    - docker login --username $DOCKER_LOGIN --password $DOCKER_PASSWORD
    # - docker run my-docker-image /script/to/run/tests
    - docker push ${DOCKER_IMAGE_TAG}:latest
  rules:
    - if: $CI_COMMIT_BRANCH == "dev"

# deploy_dev:
#   stage: deploy
#   tags:
#     - php
#   script:
#     - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
#     - eval $(ssh-agent -s)
#     - ssh-add <(echo "$SSH_PRIVATE_KEY")
#     - mkdir -p ~/.ssh
#     - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
#     # - ~/.composer/vendor/bin/envoy run deploy --commit="$CI_COMMIT_SHA"
#   environment:
#     name: dev
#     url: http://team1-prj1.dev.ia
#   when: manual
#   rules:
#     - if: $CI_COMMIT_BRANCH == "dev"