8 OpenHarmony开源鸿蒙build编译模块的原理和系统配置讲解

OpenHarmony开源鸿蒙build编译模块的原理和系统配置讲解

作者	将狼才鲸
创建日期	2024-03-12

2.1、编译模块整体描述

目的：使用这个模块里的脚本、配置来编译整个开源鸿蒙系统，生成一个操作系统可执行文件
- build模块源码地址：https://gitee.com/openharmony/build
- 基于Gn和ninja的编译构建框架（类似于CMake/QMake/Automake + Makefile）
- 使用Python文件、Python插件或者bash脚本安装依赖工具、按要求生成BUILD.gn配置文件
- build模块介绍 - 编译构建
编译的方法有三种：
- 1：运行Python源码，生成BUILD.gn并编译
- 2：运行使用Python写的插件hb，生成BUILD.gn并编译
- 3：运行bash脚本，生成BUILD.gn并编译

第一种编译，使用Python源码：

python3 build.py -p qemu_mini_system_demo@ohemu 这是编译命令
Docker编译环境
build.py文件在OpenHarmony源码文件夹的顶层，但其实顶层的文件只是一个链接：build.py -> build/build_scripts/build.py
这是build.py文件的内容：

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import os
import sys
import importlib
import subprocess

def search(findir: str, target: str) -> str or bool:
    for root, _, files in os.walk(findir):
        if target in files:
            return root
    return False

def find_top() -> str:
    cur_dir = os.getcwd()
    while cur_dir != "/":
        build_config_file = os.path.join(
            cur_dir, 'build/config/BUILDCONFIG.gn')
        if os.path.exists(build_config_file):
            return cur_dir
        cur_dir = os.path.dirname(cur_dir)

def get_python(python_relative_dir: str) -> str:
    topdir = find_top()
    if python_relative_dir is None:
        python_relative_dir = 'prebuilts/python'
    python_base_dir = os.path.join(topdir, python_relative_dir)

    if os.path.exists(python_base_dir):
        python_dir = search(python_base_dir, 'python3')
        return os.path.join(python_dir, 'python3')
    else:
        print("please execute build/prebuilts_download.sh.",
            "if you used '--python-dir', check whether the input path is valid.")
        sys.exit()

def check_output(cmd: str, **kwargs) -> str:
    process = subprocess.Popen(cmd,
                               stdout=subprocess.PIPE,
                               stderr=subprocess.STDOUT,
                               universal_newlines=True,
                               **kwargs)
    for line in iter(process.stdout.readline, ''):
        sys.stdout.write(line)
        sys.stdout.flush()

    process.wait()
    ret_code = process.returncode

    return ret_code

def build(path: str, args_list: list) -> str:
    python_dir = None
    if "--python-dir" in args_list:
        index = args_list.index("--python-dir")
        if index < len(args_list) - 1:
            python_dir = args_list[index + 1]
            del args_list[index: index + 2]
        else:
            print("-python-dir parmeter missing value.")
            sys.exit()
    python_executable = get_python(python_dir)
    cmd = [python_executable, 'build/hb/main.py', 'build'] + args_list
    return check_output(cmd, cwd=path)

def main():
    root_path = find_top()
    return build(root_path, sys.argv[1:])

if __name__ == "__main__":
    sys.exit(main())

第二种编译，运行Python插件：

HB构建工具使用指导
使用Python的插件ohos-build，使用命令为 hb set; hb build -f 安装流程为
- python3 -m pip install --user ohos-build==0.4.3 或者
- python3 -m pip install --user build/hb
- hb源码在OpenHarmony源码的./build/hb文件夹下：

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
from setuptools import setup

from resources.global_var import VERSION

LONG_DESCRIPTION = ""
WORK_PATH = os.path.abspath(os.path.dirname('__file__'))
README_PATH = os.path.join(WORK_PATH, 'README.md')
with open(README_PATH, 'r', encoding='utf-8') as FILE_DESCRIPTION:
    LONG_DESCRIPTION = FILE_DESCRIPTION.read()

setup(
    name='ohos-build',
    version=VERSION,
    author='Huawei',
    author_email='[email protected]',
    description='OHOS build command line tool',
    long_description=LONG_DESCRIPTION,
    long_description_content_type="text/markdown",
    url='https://gitee.com/openharmony/build_lite',
    license='Apache 2.0',
    python_requires='>=3.8',
    packages=['hb'],
    package_dir={'hb': 'hb'},
    package_data={'hb': ['common/config.json']},
    install_requires=[
        'prompt_toolkit==1.0.14', 'kconfiglib>=14.1.0', 'PyYAML', 'requests'
    ],
    entry_points={'console_scripts': [
        'hb=hb.__main__:main',
    ]},
)

第三种编译，使用脚本：

./build.sh --product-name {product_name}
build.sh文件在OpenHarmony源码文件夹的顶层，但其实顶层的文件只是一个链接：build.sh -> build/build_scripts/build.sh
这是build.sh文件的内容：

#!/bin/bash

set -e
set +e
echo -e "\n\033[32m\t*********Welcome to OpenHarmony!*********\033[0m\n"
echo -e "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
function check_shell_environment() {
  case $(uname -s) in 
    Linux)
          shell_result=$(/bin/sh -c 'echo ${BASH_VERSION}')
          if [ -n "${shell_result}" ]; then
            echo -e "\033[32mSystem shell: bash ${shell_result}\033[0m"
          else
            echo -e "\033[33m Your system shell isn't bash, we recommend you to use bash, because some commands may not be supported in other shells, such as pushd and shopt are not supported in dash. \n You can follow these tips to modify the system shell to bash on Ubuntu: \033[0m"
            echo -e "\033[33m [1]:Open the Terminal tool and execute the following command: sudo dpkg-reconfigure dash \n [2]:Enter the password and select <no>  \033[0m"
          fi
          ;;
    Darwin)
          echo -e "\033[31m[OHOS ERROR] Darwin system is not supported yet\033[0m"
          ;;
    *)
          echo -e "\033[31m[OHOS ERROR] Unsupported this system: $(uname -s)\033[0m"
          exit 1
  esac
}

check_shell_environment 

echo -e "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
echo -e "\033[32mCurrent time: $(date +%F' '%H:%M:%S)\033[0m"
echo -e "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
echo -e "\033[32mBuild args: $@\033[0m"
echo -e "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++"

export SOURCE_ROOT_DIR=$(cd $(dirname $0);pwd)

while [[ ! -f "${SOURCE_ROOT_DIR}/.gn" ]]; do
    SOURCE_ROOT_DIR="$(dirname "${SOURCE_ROOT_DIR}")"
    if [[ "${SOURCE_ROOT_DIR}" == "/" ]]; then
        echo -e "\033[31m[OHOS ERROR] Cannot find source tree containing $(pwd)\033[0m"
        exit 1
    fi
done

if [[ "${SOURCE_ROOT_DIR}x" == "x" ]]; then
  echo -e "\033[31m[OHOS ERROR] SOURCE_ROOT_DIR cannot be empty.\033[0m"
  exit 1
fi

case $(uname -s) in
    Darwin)
        HOST_DIR="darwin-x86"
        HOST_OS="mac"
        NODE_PLATFORM="darwin-x64"
        ;;
    Linux)
        HOST_DIR="linux-x86"
        HOST_OS="linux"
        NODE_PLATFORM="linux-x64"
        ;;
    *)
        echo "\033[31m[OHOS ERROR] Unsupported host platform: $(uname -s)\033[0m"
        RET=1
        exit $RET
esac

# set python3
PYTHON3_DIR=${SOURCE_ROOT_DIR}/prebuilts/python/${HOST_DIR}/current/
PYTHON3=${PYTHON3_DIR}/bin/python3
PYTHON=${PYTHON3_DIR}/bin/python
if [[ ! -f "${PYTHON3}" ]]; then
  echo -e "\033[31m[OHOS ERROR] Please execute the build/prebuilts_download.sh \033[0m"
  exit 1
else
  if [[ ! -f "${PYTHON}" ]]; then
    ln -sf "${PYTHON3}" "${PYTHON}"
  fi
fi

export PATH=${SOURCE_ROOT_DIR}/prebuilts/build-tools/${HOST_DIR}/bin:${PYTHON3_DIR}/bin:$PATH

# set nodejs and ohpm
EXPECTED_NODE_VERSION="14.21.1"
export PATH=${SOURCE_ROOT_DIR}/prebuilts/build-tools/common/nodejs/node-v${EXPECTED_NODE_VERSION}-${NODE_PLATFORM}/bin:$PATH
export NODE_HOME=${SOURCE_ROOT_DIR}/prebuilts/build-tools/common/nodejs/node-v${EXPECTED_NODE_VERSION}-${NODE_PLATFORM}
export PATH=${SOURCE_ROOT_DIR}/prebuilts/build-tools/common/oh-command-line-tools/ohpm/bin:$PATH
echo "[OHOS INFO] Current Node.js version is $(node -v)"
NODE_VERSION=$(node -v)
if [ "$NODE_VERSION" != "v$EXPECTED_NODE_VERSION" ]; then
  echo -e "\033[31m[OHOS ERROR] Node.js version mismatch. Expected $EXPECTED_NODE_VERSION but found $NODE_VERSION\033[0m" >&2
  exit 1
fi
echo -e "\033[32m[OHOS INFO] Node.js version check passed!\033[0m"
npm config set registry https://repo.huaweicloud.com/repository/npm/
npm config set @ohos:registry https://repo.harmonyos.com/npm/
npm config set strict-ssl false
npm config set lockfile false
cat $HOME/.npmrc | grep 'lockfile=false' > /dev/null || echo 'lockfile=false' >> $HOME/.npmrc > /dev/null

function init_ohpm() {
  TOOLS_INSTALL_DIR="${SOURCE_ROOT_DIR}/prebuilts/build-tools/common"
  pushd ${TOOLS_INSTALL_DIR} > /dev/null
    if [[ ! -f "${TOOLS_INSTALL_DIR}/oh-command-line-tools/ohpm/bin/ohpm" ]]; then
      echo "[OHOS INFO] download oh-command-line-tools"
      wget https://contentcenter-vali-drcn.dbankcdn.cn/pvt_2/DeveloperAlliance_package_901_9/68/v3/r-5H8I7LT9mBjSFpSOY0Sg/ohcommandline-tools-linux-2.1.0.6.zip\?HW-CC-KV\=V1\&HW-CC-Date\=20231027T004601Z\&HW-CC-Expire\=315360000\&HW-CC-Sign\=A4D5E1A29C1C6962CA65592C3EB03ED363CE664CBE6C5974094064B67C34325E -O ohcommandline-tools-linux.zip
      unzip ohcommandline-tools-linux.zip
    fi
    OHPM_HOME=${TOOLS_INSTALL_DIR}/oh-command-line-tools/ohpm
    chmod +x ${OHPM_HOME}/bin/init
    ${OHPM_HOME}/bin/init > /dev/null
    echo "[OHOS INFO] Current ohpm version is $(ohpm -v)"
    ohpm config set registry https://repo.harmonyos.com/ohpm/
    ohpm config set strict_ssl false
    ohpm config set log_level debug
  popd > /dev/null
  if [[ -d "$HOME/.hvigor" ]]; then
    rm -rf $HOME/.hvigor/daemon $HOME/.hvigor/wrapper
  fi
  mkdir -p $HOME/.hvigor/wrapper/tools
  echo '{"dependencies": {"pnpm": "7.30.0"}}' > $HOME/.hvigor/wrapper/tools/package.json
  pushd $HOME/.hvigor/wrapper/tools > /dev/null
    echo "[OHOS INFO] installing pnpm..."
    npm install --silent > /dev/null
  popd > /dev/null
  mkdir -p $HOME/.ohpm
  echo '{"devDependencies":{"@ohos/hypium":"1.0.6"}}' > $HOME/.ohpm/oh-package.json5
  pushd $HOME/.ohpm > /dev/null
    echo "[OHOS INFO] installing hypium..."
    ohpm install > /dev/null
  popd > /dev/null
}

if [[ "$*" != *ohos-sdk* ]]; then
  echo "[OHOS INFO] Ohpm initialization started..."
  init_ohpm
  if [[ "$?" -ne 0 ]]; then
    echo -e "\033[31m[OHOS ERROR] ohpm initialization failed!\033[0m"
    exit 1
  fi
  echo -e "\033[32m[OHOS INFO] ohpm initialization successful!\033[0m"
fi
echo -e "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n"

echo -e "\033[32m[OHOS INFO] Start building...\033[0m\n"
function build_sdk() {
    ROOT_PATH=${SOURCE_ROOT_DIR}
    SDK_PREBUILTS_PATH=${ROOT_PATH}/prebuilts/ohos-sdk

    pushd ${ROOT_PATH} > /dev/null
      echo -e "[OHOS INFO] building the latest ohos-sdk..."
      ./build.py --product-name ohos-sdk $ccache_args $xcache_args --load-test-config=false --get-warning-list=false --stat-ccache=false --compute-overlap-rate=false --deps-guard=false --generate-ninja-trace=false --gn-args skip_generate_module_list_file=true sdk_platform=linux ndk_platform=linux use_cfi=false use_thin_lto=false enable_lto_O0=true sdk_check_flag=false enable_ndk_doxygen=false archive_ndk=false sdk_for_hap_build=true
      if [[ "$?" -ne 0 ]]; then
        echo -e "\033[31m[OHOS ERROR] ohos-sdk build failed! You can try to use '--no-prebuilt-sdk' to skip the build of ohos-sdk.\033[0m"
        exit 1
      fi
      if [ -d "${ROOT_PATH}/prebuilts/ohos-sdk/linux" ]; then
          rm -rf ${ROOT_PATH}/prebuilts/ohos-sdk/linux
      fi
      mkdir -p ${SDK_PREBUILTS_PATH}
      mv ${ROOT_PATH}/out/sdk/ohos-sdk/linux ${SDK_PREBUILTS_PATH}/
      mkdir -p ${SDK_PREBUILTS_PATH}/linux/native
      mv ${ROOT_PATH}/out/sdk/sdk-native/os-irrelevant/* ${SDK_PREBUILTS_PATH}/linux/native/
      mv ${ROOT_PATH}/out/sdk/sdk-native/os-specific/linux/* ${SDK_PREBUILTS_PATH}/linux/native/
      pushd ${SDK_PREBUILTS_PATH}/linux > /dev/null
        api_version=$(grep apiVersion toolchains/oh-uni-package.json | awk '{print $2}' | sed -r 's/\",?//g') || api_version="11"
        mkdir -p $api_version
        for i in */; do
            if [ -d "$i" ] && [ "$i" != "$api_version/" ]; then
                mv $i $api_version
            fi
        done
      popd > /dev/null
    popd > /dev/null
}
if [[ ! -d "${SOURCE_ROOT_DIR}/prebuilts/ohos-sdk/linux" && "$*" != *ohos-sdk* && "$*" != *"--no-prebuilt-sdk"* || "${@}" =~ "--prebuilt-sdk" ]]; then
  echo -e "\033[33m[OHOS INFO] The OHOS-SDK was not detected, so the SDK compilation will be prioritized automatically. You can also control whether to execute this process by using '--no-prebuilt-sdk' and '--prebuilt-sdk'.\033[0m"
  if [[ "${@}" =~ "--ccache=false" || "${@}" =~ "--ccache false" ]]; then
    ccache_args="--ccache=false"
  else
    ccache_args="--ccache=true"
  fi
  if [[ "${@}" =~ "--xcache=true" || "${@}" =~ "--xcache true" || "${@}" =~ "--xcache" ]]; then
    xcache_args="--xcache=true"
  else
    xcache_args="--xcache=false"
  fi
  build_sdk
  if [[ "$?" -ne 0 ]]; then
    echo -e "\033[31m[OHOS ERROR] ohos-sdk build failed, please remove the out/sdk directory and try again!\033[0m"
    exit 1
  fi
fi

${PYTHON3} ${SOURCE_ROOT_DIR}/build/scripts/tools_checker.py

flag=true
args_list=$@
for var in $@
do
  OPTIONS=${var%%=*}
  PARAM=${var#*=}
  if [[ "$OPTIONS" == "using_hb_new" && "$PARAM" == "false" ]]; then
    flag=false
    ${PYTHON3} ${SOURCE_ROOT_DIR}/build/scripts/entry.py --source-root-dir ${SOURCE_ROOT_DIR} $args_list
    break
  fi
done
if [[ ${flag} == "true" ]]; then
  ${PYTHON3} ${SOURCE_ROOT_DIR}/build/hb/main.py build $args_list
fi

if [[ "$?" -ne 0 ]]; then
    echo -e "\033[31m=====build ${product_name} error=====\033[0m"
    exit 1
fi
echo -e "\033[32m=====build ${product_name} successful=====\033[0m"

date +%F' '%H:%M:%S
echo "++++++++++++++++++++++++++++++++++++++++"

2.2 OpenHarmony编译前的系统配置

进行过Linux内核编程的就知道，make menuconfig里面有很多配置，一个操作系统要进行定制，编译静态库动态库，就需要在编译前进行各种配置。
- 开源鸿蒙配置文件说明
OpenHarmony的配置方法：
- 1：可以使用gn语法规则自定义编写模块内的BUILD.gn文件（有模块模板）
- 2：可以手动修改bundle.json、ohos.build等文件
- 3：编译时直接把配置输进去，如：./build.sh --product-name ohos-sdk --ccache
使用方法：修改.json和.gn文件
鸿蒙build编译子模块中有个顶层的bundle.json配置：

{
  "name": "@ohos/build_framework",
  "description": "build framework",
  "version": "4.0.2",
  "license": "Apache License 2.0",
  "homePage":"https://gitee.com/openharmony",
  "repository":"https://gitee.com/openharmony/build",
  "supplier": "Organization: OpenHarmony",
  "publishAs": "code-segment",
  "segment": {
      "destPath": "build"
  },
 "readmePath":{
      "zh": "README_zh.md"
 },
 "dirs": {},
 "scripts": {
      "install": "cd ${DEP_BUNDLE_BASE} && ln -snf build/build_scripts/build.py build.py && ln -snf build/build_scripts/build.sh build.sh && ln -snf build/core/gn/dotfile.gn .gn"
 },
  "component": {
    "name": "build_framework",
    "description": "build_framework component set",
    "subsystem": "build",
    "features": [],
    "adapted_system_type": [
      "standard",
      "small",
      "mini"
    ],
    "rom": "0KB",
    "ram": "0KB",
    "deps": {
      "components": [],
        "third_party": [
        "musl",
        "markupsafe",
        "jinja2",
        "e2fsprogs",
        "f2fs-tools"
      ]
    },
    "build": {
      "sub_component": [
        "//build/common:common_packages",
        "//build/rust:default",
        "//third_party/f2fs-tools:f2fs-tools_host_toolchain"
      ],
      "inner_api": [
        {
          "name": "//build/rust:libstd.dylib.so",
          "header": {
            "header_base": [],
            "header_files": []
          }
        },
        {
          "name": "//build/rust:libtest.dylib.so",
          "header": {
            "header_files": [],
            "header_base": []
          }
        }
      ],
      "test": [
        "//build/rust/tests:tests"
      ]
    }
  }
}

./core/gn下有个BUILD.gn文件：

print("root_out_dir=$root_out_dir")
print("root_build_dir=$root_build_dir")
print("root_gen_dir=$root_gen_dir")
print("current_toolchain=$current_toolchain")
print("host_toolchain=$host_toolchain")

import("//build/ohos_var.gni")

# gn target defined
if (product_name == "ohos-sdk") {
  group("build_ohos_sdk") {
    deps = [
      "//build/ohos/ndk:ohos_ndk",
      "//build/ohos/sdk:ohos_sdk",
      "//build/ohos/sdk:ohos_sdk_verify",
    ]
  }
} else if (product_name == "arkui-x") {
  group("arkui_targets") {
    deps = [ "//build_plugins/sdk:arkui_cross_sdk" ]
  }
} else {
  group("make_all") {
    deps = [
      ":make_inner_kits",
      ":packages",
    ]
    if (is_standard_system && !is_llvm_build) {
      # Lite system uses different packaging scheme, which is called in hb.
      # So skip images for lite system since it's the mkimage
      # action for standard system.
      deps += [ ":images" ]
    }
  }

  if (!is_llvm_build) {
    group("images") {
      deps = [ "//build/ohos/images:make_images" ]
    }
  }

  group("packages") {
    deps = [ "//build/ohos/packages:make_packages" ]
  }

  group("make_inner_kits") {
    deps = [ "$root_build_dir/build_configs:inner_kits" ]
  }

  group("build_all_test_pkg") {
    testonly = true
    if (!is_llvm_build) {
      deps = [
        "$root_build_dir/build_configs:parts_test",
        "//test/testfwk/developer_test:make_temp_test",
      ]
    }
  }

  group("make_test") {
    testonly = true
    deps = [
      "//build/ohos/packages:build_all_test_pkg",
      "//build/ohos/packages:package_testcase",
      "//build/ohos/packages:package_testcase_mlf",
    ]
    if (archive_component) {
      deps += [ "//build/ohos/testfwk:archive_testcase" ]
    }
  }
}

3、boot代码

上电第一行的boot代码，硬件复位中断，RAM零地址的代码

8 OpenHarmony开源鸿蒙build编译模块的原理和系统配置讲解

OpenHarmony开源鸿蒙build编译模块的原理和系统配置讲解

2.1、编译模块整体描述

第一种编译，使用Python源码：

第二种编译，运行Python插件：

第三种编译，使用脚本：

2.2 OpenHarmony编译前的系统配置

3、boot代码

悦读