Docker 编译OpenHarmony 4.0 release

一、背景介绍

1.1、环境配置

编译环境：Ubuntu 20.04
OpenHarmony版本：4.0 release
平台设备：RK3568

OpenHarmony 3.2更新至OpenHarmony 4.0后，公司服务器无法编译通过，总是在最后几十个文件时报错,错误码4000：

[OHOS ERROR] ld.lld: error: undefined symbol: __aarch64_cas4_acq_rel
[OHOS ERROR] >>> referenced by hdf_vnode_adapter.c:294 (/home/xxx/OHOS/OpenHarmony_master/out/kernel/OBJ/linux-5.10/../../../../drivers/hdf_core/framework/core/adapter/vnode/src/hdf_vnode_adapter.c:294)
[OHOS ERROR] >>> vmlinux.o:(HdfVNodeAdapterIoctl)
[OHOS ERROR] >>> referenced by hdf_vnode_adapter.c:303 (/home/xxx/OHOS/OpenHarmony_master/out/kernel/OBJ/linux-5.10/../../../../drivers/hdf_core/framework/core/adapter/vnode/src/hdf_vnode_adapter.c:303)
[OHOS ERROR] >>> vmlinux.o:(HdfVNodeAdapterIoctl)
[OHOS ERROR] >>> referenced by hdf_vnode_adapter.c:428 (/home/xxx/OHOS/OpenHarmony_master/out/kernel/OBJ/linux-5.10/../../../../drivers/hdf_core/framework/core/adapter/vnode/src/hdf_vnode_adapter.c:428)
[OHOS ERROR] >>> vmlinux.o:(HdfDeviceSendEventToClient)
[OHOS ERROR] >>> referenced 1 more times
[OHOS ERROR] BTF .btf.vmlinux.bin.o
[OHOS ERROR] pahole: .tmp_vmlinux.btf: No such file or directory
[OHOS ERROR] LD .tmp_vmlinux.kallsyms1
[OHOS ERROR] ld.lld: error: .btf.vmlinux.bin.o: unknown file type
[OHOS ERROR] make[2]: *** [/home/xxx/OHOS/OpenHarmony_master/out/kernel/src_tmp/linux-5.10/Makefile:1236: vmlinux] Error 1
[OHOS ERROR] make[1]: *** [arch/arm64/Makefile:208: rk3568-toybrick-x0-linux.img] Error 2
[OHOS ERROR] make[1]: Leaving directory '/home/xxx/OHOS/OpenHarmony_master/out/kernel/OBJ/linux-5.10'
[OHOS ERROR] make: *** [Makefile:192: __sub-make] Error 2
[OHOS ERROR] Traceback (most recent call last):
...
[OHOS ERROR] raise OHOSException('ninja phase failed', '4000')
[OHOS ERROR] exceptions.ohos_exception.OHOSException: ninja phase failed
[OHOS ERROR]
[OHOS ERROR] Code: 4000
[OHOS ERROR]
[OHOS ERROR] Reason: ninja phase failed
[OHOS ERROR]
[OHOS ERROR] Solution: Please check the compile log at out/{compiling product}/build.log, If you could analyze build logs.
[OHOS ERROR] Or you can try the following steps to solve this problem:
[OHOS ERROR] 1. cd to OHOS root path
[OHOS ERROR] 2. run 'hb clean --all' or 'rm -rf out build/resources/args/.json'.
[OHOS ERROR] 3. repo sync
[OHOS ERROR] 4. repo forall -c 'git lfs pull'
[OHOS ERROR] 5. bash build/prebuilts_download.sh
[OHOS ERROR] 6. rebuild your product or component
[OHOS ERROR] 7. check if any environment variables required by the building process are not set in build/compile_env_allowlist.json
[OHOS ERROR]
[OHOS ERROR] If you still cannot solve this problem, you could post this problem on:
[OHOS ERROR] https://gitee.com/openharmony/build/issues
[OHOS ERROR]
=====build error=====

经分析尝试：
1、相同的步骤和命令，wsl2 编译OpenHarmony 4.0 r正常。
2、服务器使用sudo编译正常，但由于sudo使用root权限，会影响项目编译链等配置，故不能作为解决方案。

于是尝试在服务器中使用docker，通过隔离环境编译OpenHarmony 4.0。

二、docker安装与配置

docker
在Ubuntu服务器中安装和配置docker

2.1、docker安装

$ sudo apt-get update
$ sudo apt-get upgrade

$ sudo apt install docker.io
#安装后通过查看docker版本验证是否安装成功
$ docker --version 或 docker -v
Docker version 24.0.5, build 24.0.5-0ubuntu1~20.04.1

2.2、获取docker 镜像

获取docker镜像，以ubuntu:20.04为例
注意：此过程较漫长，请耐心等待~

$ docker pull ubuntu:20.04

#创建好后可通过image命令查看
$ docker images 或docker image ls
REPOSITORY                                                               TAG       IMAGE ID       CREATED        SIZE
ubuntu                                                                   20.04     f78909c2b360   8 days ago     72.8MB
swr.cn-south-1.myhuaweicloud.com/openharmony-docker/docker_oh_standard   3.2       ae33c06f45c5   6 months ago   5.3GB

2.3、创建docker容器

#进入需要与docker映射的目录，后续基于此目录的修改，容器和服务会同步。
$ cd ~/docker/
$ docker run -it -v $(pwd):/home/zhangsan --name ohos ubuntu:20.04

说明：命令表示创建一个名为ohos，系统镜像为ubuntu 20.04的容器；这个容器运行在交互模式下，并且将当前目录映射到容器的/home/zhangsan目录。
注意：zhangsan需与后续要在ohos创建的非root用户名保持一致，否则后续下拉和编译代码会出现权限问题。

创建好ohos容器，默认以root用户进入。

#退出root用户
$ exit
# 启动ohos容器
$ docker container start ohos
#查看docker容器
$ docker container ls
CONTAINER ID   IMAGE          COMMAND       CREATED        STATUS        PORTS     NAMES
ab7d67807966   ubuntu:20.04   "/bin/bash"   29 hours ago   Up 29 hours             ohos

2.4、ohos容器中创建非root用户

由于docker 容易与宿主系统共享一个内核，容器中的用户ID和组ID是映射到宿主系统的；默认情况下，容器内的所有进程都以root用户身份运行。这意味着容器内的进程具有与宿主系统中的root用户相同的权限。为了提高安全性，建议在容器中使用非root用户运行进程。

#运行ohos容器（默认root用户）
$ docker exec -it ohos bash
$ apt-get update
$ apt-get upgrade
$ apt install -y sudo
$ apt install -y vim

#添加非root用户zhangsan
$ useradd -m zhangsan
#设置用户密码，123456
$ passwd zhangsan
#设置zhangsan用户组
$ usermod -aG zhangsan zhangsan

#sudoers添加zhangsan
$ vim /etc/sudoers	#文末添加一行zhangsan	ALL=(ALL)	ALL
#设置/home/zhangsan目录用户和组，需与用户名保持一致
$ chown -R zhangsan:zhangsan /home/zhangsan
$ exit

注意：zhangsan的uid和gid需与宿主系统自己的uid和gid保持一致，否则后续下载源码后，在宿主系统中自己无权限修改。
如下zhangsan 与wangwu的id匹配一致，为1003

#ohos容器zhangsan 用户名,uid,gid和组信息
$ whoami && id
uid=1003(zhangsan) gid=1003(zhangsan) groups=1003(zhangsan)

#宿主系统自己账户（wangwu）用户名,uid,gid和组信息
$ whoami && id
wangwu
uid=1003(wangwu) gid=1003(wangwu) groups=1003(wangwu)

若ohos容器中zhangsan的uid与wangwu的不一致(通常zhangsan.uid < wangwu.uid）,可在容器中添加任意用户，uid号以1000开始累加，最终使得zhangsan.uid = wangwu.uid即可

#由于uid不匹配，先删除zhangsan用户
userdel zhangsan

# 添加随意用户 tom,jerry
$ useradd -M tom
$ useradd -M jerry

#查看所有用户
cat /etc/passwd
...
tom:x:1001:1001::/home/tom:/bin/sh
jerry:x:1002:1002::/home/jerry:/bin/sh

#此时再次添加zhangsan用户，其uid则为1003

三、OpenHarmony 4.0 源码下载及编译

3.1、配置ohos容器编译环境

#以zhangsan用户运行ohos容器
$ docker exec --user zhangsan -it ohos bash

#替换软件源
$ sudo mv /etc/apt/sources.list /etc/apt/sources.list.bak
$ sudo vim /etc/apt/sources.list	#sources.list替换以下内容

deb http://mirrors.ustc.edu.cn/ubuntu/ focal main restricted universe multiverse
deb-src http://mirrors.ustc.edu.cn/ubuntu/ focal main restricted universe multiverse

deb http://mirrors.ustc.edu.cn/ubuntu/ focal-security main restricted universe multiverse
deb-src http://mirrors.ustc.edu.cn/ubuntu/ focal-security main restricted universe multiverse

deb http://mirrors.ustc.edu.cn/ubuntu/ focal-updates main restricted universe multiverse
deb-src http://mirrors.ustc.edu.cn/ubuntu/ focal-updates main restricted universe multiverse

deb http://mirrors.ustc.edu.cn/ubuntu/ focal-backports main restricted universe multiverse
deb-src http://mirrors.ustc.edu.cn/ubuntu/ focal-backports main restricted universe multiverse

## Not recommended
# deb http://mirrors.ustc.edu.cn/ubuntu/ focal-proposed main restricted universe multiverse
# deb-src http://mirrors.ustc.edu.cn/ubuntu/ focal-proposed main restricted universe multiverse

更新
$ sudo apt-get update
$ sudo apt-get upgrade

#将Shell环境修改为bash
$ sudo dpkg-reconfigure dash  #select no

#部署编译环境，安装工具包
$ sudo apt-get install -y gcc-arm-linux-gnueabi gcc-9-arm-linux-gnueabi
$ sudo apt-get install -y python python3 python3-pip binutils binutils-dev git git-lfs gnupg flex bison gperf build-essential zip curl zlib1g-dev   libc6-dev-i386 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z1-dev ccache libgl1-mesa-dev libxml2-utils xsltproc unzip m4 bc gnutls-bin python3.8 python3-pip ruby genext2fs device-tree-compiler make libffi-dev e2fsprogs pkg-config perl openssl libssl-dev libelf-dev libdwarf-dev u-boot-tools mtd-utils cpio doxygen liblz4-tool openjdk-8-jre gcc g++ texinfo dosfstools mtools default-jre default-jdk libncurses5 apt-utils wget scons python3.8-distutils tar rsync git-core libxml2-dev lib32z-dev grsync xxd libglib2.0-dev libpixman-1-dev kmod jfsutils reiserfsprogs xfsprogs squashfs-tools pcmciautils quota ppp libtinfo-dev libtinfo5 libncurses5-dev libncursesw5 libstdc++6 gcc-arm-none-eabi vim ssh locales libxinerama-dev libxcursor-dev libxrandr-dev libxi-dev
#上一步命令执行过程需手动选择时区配置：6.Asia/70.Shanghai

#配置pip软件包更新源
$ mkdir ~/.pip
$ pip3 config set global.index-url https://mirrors.huaweicloud.com/repository/pypi/simple
$ pip3 config set global.trusted-host mirrors.huaweicloud.com
$ pip3 config set global.timeout 120

#配置git
git config --global user.email "jerry@163.com"
git config --global user.name "jerry"
git config --global credential.helper store
git config --global --add safe.directory "*"

#安装repo
wget https://gitee.com/oschina/repo/raw/fork_flow/repo-py3
sudo mv repo-py3 /usr/local/bin/repo
sudo chmod a+x /usr/local/bin/repo
pip3 install -i https://repo.huaweicloud.com/repository/pypi/simple requests

3.2、下拉OpenHarmony 4.0r源码及编译

$ mkdir OpenHarmony_4.0r && cd OpenHarmony_4.0r
$ repo init -u https://gitee.com/openharmony/manifest -b OpenHarmony-4.0-Release --no-repo-verify
$ repo sync -c && repo forall -c 'git lfs pull'
$ bash build/prebuilts_download.sh
$ ./build.sh -p rk3568	#编译成功，结果如下
...
Please modify according to README.md
[OHOS INFO] rk3568 build success
[OHOS INFO] Cost time:  0:14:17
=====build  successful=====
2023-12-21 21:29:18
++++++++++++++++++++++++++++++++++++++++

宿主系统可见openharmony 4.0r代码已拉下，并有权限修改

$ tree -L 2 ~/docker/
docker/
├── OpenHarmony_4.0r
│   ├── applications
│   ├── arkcompiler
│   ├── base
│   ├── build
│   ├── build.py -> build/build_scripts/build.py
│   ├── build.sh -> build/build_scripts/build.sh
│   ├── commonlibrary
│   ├── developtools
│   ├── device
│   ├── docs
│   ├── domains
│   ├── drivers
│   ├── foundation
...

#创建文件正常
$ touch ~/docker/OpenHarmony_4.0r/test