Porting/Chromium (Old)

From RCS Wiki
Revision as of 12:51, 18 February 2019 by Shawnanastasio (talk | contribs) (Add boringssl BUILD.gn patch)
Jump to navigation Jump to search

Further information on: Work specific to Big Endian support

Further information on: Electron support

  • What to port to ppc
    • nacl (Not porting since it's been deprecated by Google.)

# Install Ubuntu 18.04 (Bionic Beaver) LTS ppc64le in VM with good amount of CPUs and ~16GB RAM, the more CPUs the more RAM, and around 200GB disk space

sudo apt update
sudo apt full-upgrade
sudo apt install git vim cmake python libcups2-dev pkg-config libnss3-dev libssl-dev libglib2.0-dev libgnome-keyring-dev libpango1.0-dev libdbus-1-dev libatk1.0-dev libatk-bridge2.0-dev libgtk-3-dev libkrb5-dev libpulse-dev libxss-dev re2c subversion curl libasound2-dev libpci-dev mesa-common-dev gperf bison nodejs uuid-dev clang-format

##begin alternate - use a xenial-chroot instead of a VM
#Create chroot
sudo debootstrap --include=software-properties-common xenial xenial-chroot http://us.ports.ubuntu.com/ubuntu-ports/ubuntu-ports/ 

#must be mapped for chroot to function properly
sudo mount -t proc none xenial-chroot/proc/
sudo mount -o bind /sys/ xenial-chroot/sys/
sudo mount -o bind /dev/shm/ xenial-chroot/dev/shm/

#enter chroot
sudo chroot xenial-chroot

#add additional apt sources
echo "deb http://us.ports.ubuntu.com/ubuntu-ports/ubuntu-ports xenial-updates main" >> /etc/apt/sources.list
echo "deb http://us.ports.ubuntu.com/ubuntu-ports/ubuntu-ports xenial universe" >> /etc/apt/sources.list
echo "deb http://us.ports.ubuntu.com/ubuntu-ports/ubuntu-ports xenial-updates universe" >> /etc/apt/sources.list

#update and add software as mentioned above
apt update
apt full-upgrade
apt install <packages listed above>

#get nodejs and libatspi2.0-dev from bionic:
echo "deb http://us.ports.ubuntu.com/ubuntu-ports/ubuntu-ports bionic main universe" >> /etc/apt/sources.list
apt update
apt-get install -t bionic nodejs libatspi2.0-dev

#add chroot user
useradd -m build-user -s /bin/bash
su build-user
echo 'cd' >> ~/.bashrc
cd
##end alternate - use a xenial-chroot instead of a VM


cd
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git

echo 'export PATH="$PATH:${HOME}/ninja"' >> ~/.bashrc
echo 'export PATH="$PATH:${HOME}/depot_tools"' >> ~/.bashrc
echo 'export VPYTHON_BYPASS="manually managed python not supported by chrome operations"' >> ~/.bashrc
echo 'export GYP_DEFINES="disable_nacl=1"' >> ~/.bashrc
. ~/.bashrc

git clone git://github.com/ninja-build/ninja.git
cd ninja
git checkout release
./configure.py --bootstrap
which ninja # ensure it is right ninja bin, else figure out a way so it is the right one that comes up


cd
git clone https://gn.googlesource.com/gn

export CC=gcc
export CXX=g++
export AR=ar

cd gn
python build/gen.py --no-sysroot
cat out/build.ninja | sed s/-stdlib=libstdc++//g | tee out/build.ninja
ninja -C out

unset CXX
unset CC
unset AR

# Setup clang (HIGHLY RECOMMENDED)
# Building chromium with gcc is an uphill battle as the devs only test with clang.
# It is possible to set up a local clang compiler with the exact revision chromium expects by doing the following:

# Check out LLVM (rev 353250)
svn checkout --force https://llvm.org/svn/llvm-project/llvm/trunk@353250 llvm

# Check out Clang (rev 353250)
svn checkout --force https://llvm.org/svn/llvm-project/cfe/trunk@353250 llvm/tools/clang

# Check out compiler-rt (rev 353250)
svn checkout --force https://llvm.org/svn/llvm-project/compiler-rt/trunk@353250 llvm/compiler-rt

# Make build directory and build
mkdir llvm_build
cd llvm_build
cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD="PowerPC" -G "Unix Makefiles" ../llvm
make -j64 # Replace with no. of threads in your box

# Work around libatomic problems under Clang
# See https://github.com/cquery-project/cquery/issues/382 for a related problem in a different project
cd /usr/lib/powerpc64le-linux-gnu
ln -s libatomic.so.1 libatomic.so

# Setup and build chromium tree
cd
mkdir chromium
cd chromium

# The following command will fail with a message "failed to resolve package version".
# Once it does, simply continue following the instructions. The subsequent patches will resolve the issue.
fetch --no-history --nohooks chromium

curl "https://wiki.raptorcs.com/w/images/4/45/Src-DEPS.patch" | patch -p1 src/DEPS # If this patchfile stops working, ignore the error, see "Patchfiles uploaded"
curl "https://wiki.raptorcs.com/w/images/1/10/Binutils-download.py-PPC.patch" | patch -p1 src/third_party/binutils/download.py
cd src && curl "https://wiki.raptorcs.com/w/images/d/d2/0001-build-download_nacl_toolchains.py-Honor-disable_nacl.patch" | patch -p1 && cd ..

gclient sync
gclient runhooks

mv src/buildtools/linux64/gn src/buildtools/linux64/gn.old
ln -s $HOME/gn/out/gn src/buildtools/linux64/gn

cd src

# build system patches
curl "https://wiki.raptorcs.com/w/images/6/69/Buildtools-third_party-libc%2B%2B-trunk-include-thread-PPC.patch" | patch -p1 buildtools/third_party/libc++/trunk/include/thread
curl "https://wiki.raptorcs.com/w/images/5/5c/0006-build-toolchain-allow-clang-on-ppc64.patch"                  | patch -p1

# breakpad fix
cd third_party/breakpad/breakpad
curl "https://wiki.raptorcs.com/w/images/f/f7/0001-Implement-ppc64-support-on-linux.patch"                      | patch -p1
cd ../../..

# crashpad fix
cd third_party/crashpad/crashpad
curl "https://wiki.raptorcs.com/w/images/8/88/0001-Implement-support-for-PPC64-on-Linux.patch"                  | patch -p1
cd ../../..
curl "https://wiki.raptorcs.com/w/images/6/6e/0002-Include-cstddef-to-fix-build.patch"                          | patch -p1

# webrtc patches
curl "https://wiki.raptorcs.com/w/images/b/b2/Rtc_base-system-arch.h-PPC.patch" | patch -p1 third_party/webrtc/rtc_base/system/arch.h
curl "https://wiki.raptorcs.com/w/images/9/96/Modules-desktop_capture-differ_block.cc-PPC.patch" | patch -p1 third_party/webrtc/modules/desktop_capture/differ_block.cc

# sandbox patches
curl "https://wiki.raptorcs.com/w/images/a/ab/0001-sandbox-linux-bpf_dsl-Update-syscall-ranges-for-ppc6.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/3/30/0002-sandbox-linux-bpf_dsl-Modify-seccomp_macros-to-add-s.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/6/60/0003-sandbox-linux-system_headers-Update-linux-seccomp-he.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/e/ec/0004-sandbox-linux-system_headers-Update-linux-signal-hea.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/b/b3/0005-sandbox-linux-seccomp-bpf-Add-ppc64-syscall-stub.patch"     | patch -p1
curl "https://wiki.raptorcs.com/w/images/f/f5/Sandbox-linux-services-credentials.cc-PPC.patch"                 | patch -p1
curl "https://wiki.raptorcs.com/w/images/b/be/0001-sandbox-Enable-seccomp_bpf-for-ppc64.patch"                 | patch -p1
curl "https://wiki.raptorcs.com/w/images/3/38/0001-sandbox-linux-Implement-partial-support-for-ppc64-sy.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/b/bb/0001-sandbox-linux-seccomp-bpf-helpers-Fix-TCGETS-declara.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/e/e7/0001-sandbox-linux-seccomp-bpf-helpers-Skip-vserver-sysca.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/4/43/0001-sandbox-linux-Update-syscall-helpers-lists-for-ppc64.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/5/5a/0001-sandbox-linux-Update-IsSyscallAllowed-in-broker_proc.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/0/0d/0001-linux-seccomp-bpf-ppc64-glibc-workaround-in-SIGSYS-h.patch" | patch -p1

# service_manager patches
curl "https://wiki.raptorcs.com/w/images/c/c9/0001-services-service_manager-sandbox-linux-Fix-TCGETS-de.patch" | patch -p1

# third_party patches
curl "https://wiki.raptorcs.com/w/images/8/8a/0001-third_party-angle-Include-missing-header-cstddef-in-.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/f/fc/0001-third_party-lss-Don-t-look-for-mmap2-on-ppc64.patch"        | patch -p1
curl "https://wiki.raptorcs.com/w/images/8/8d/0001-DONTMERGE-third_party-node-Use-system-nodejs-binary.patch"  | patch -p1
curl "https://wiki.raptorcs.com/w/images/0/0b/0002-third_party-closure_compiler-Remove-useless-JVM-flag.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/5/5f/0001-third_party-blink-Fix-build-on-ppc64.patch"                 | patch -p1
curl "https://wiki.raptorcs.com/w/images/8/84/0001-components-update_client-Fix-building-on-ppc64.patch"       | patch -p1
curl "https://wiki.raptorcs.com/w/images/0/02/0001-Reenable-VSX-in-libpng.patch"                               | patch -p1
curl "https://wiki.raptorcs.com/w/images/2/28/0001-third_party-boringssl-Properly-detect-ppc64le-in-BUI.patch" | patch -p1

# page size fixes
curl "https://wiki.raptorcs.com/w/images/f/f3/0001-base-allocator-Use-64k-page-sizes-on-ppc64.patch"           | patch -p1
curl "https://wiki.raptorcs.com/w/images/5/5c/0001-Apply-blink-64k-page-workaround-to-ppc64-systems.patch"     | patch -p1

# libvpx fix
curl "https://wiki.raptorcs.com/w/images/8/80/0001-third_party-libvpx-Properly-generate-gni-on-ppc64.patch"    | patch -p1
# VP9 rendering is broken with the ppc64 VSX code in libvpx. You can optionally apply the following patch to work around it:
curl "https://wiki.raptorcs.com/w/images/5/5e/HACK-third_party-libvpx-use-generic-gnu.patch"                   | patch -p1

cd third_party/libvpx
mkdir source/config/linux/ppc64
./generate_gni.sh
cd ../../

# pdfium fixes
cd third_party/pdfium
# The following patch is no longer needed on recent chromium revisions and will fail to apply
#curl "https://wiki.raptorcs.com/w/images/4/41/0001-Add-spin-yield-support-for-ppc64.patch"                      | patch -p1

curl "https://wiki.raptorcs.com/w/images/2/25/0001-pdfium-allocator-Use-64k-page-sizes-on-ppc64.patch"          | patch -p1
cd ../..

# v8 fixes
cd v8
curl "https://wiki.raptorcs.com/w/images/3/3c/0001-DONTMERGE-Disable-v8-unit-tests.patch"                       | patch -p1
curl "https://wiki.raptorcs.com/w/images/b/b0/0001-Force-baseline-POWER8-AltiVec-VSX-CPU-features-when-.patch"  | patch -p1
# The following two patches are upstream, and pending upstream respectively. Ignore them if they don't apply.
curl "https://wiki.raptorcs.com/w/images/9/9c/0001-PPC-s390-iwyu-Remove-include-heap-inl.h-counters.h.patch"    | patch -p1
curl "https://wiki.raptorcs.com/w/images/c/c2/0001-Fix-build-issues-on-64k-systems.patch"                       | patch -p1
cd ..

# database fix
curl "https://wiki.raptorcs.com/w/images/d/df/0001-Properly-detect-little-endian-PPC64-systems.patch"           | patch -p1

# blink fix
curl "https://wiki.raptorcs.com/w/images/a/aa/0001-third_party-blink-Add-SaveRegisters-implementation-w.patch"  | patch -p1

### GCC ONLY! (NO LONGER NEEDED, MERGED UPSTREAM)

# quick fix https://bugs.chromium.org/p/chromium/issues/detail?id=882347&can=2&q=gcc&colspec=ID%20Pri%20M%20Stars%20ReleaseBlock%20Component%20Status%20Owner%20Summary%20OS%20Modified
#git checkout 7b4f22bd25461cb32236c07699b029c7411ba494 -- net/third_party/quic/core/frames/quic_frame.h

# blink gcc fix https://bugs.chromium.org/p/chromium/issues/detail?id=882161#
#curl "https://wiki.raptorcs.com/w/images/d/de/Chromium-blink-snapcontainerdata.patch" | patch -p1

### END GCC ONLY

gn args out/Default

# PASTE IN EDITOR

# Release mode
is_component_build = false
is_debug = false

# Disable broken features
enable_nacl = false
treat_warnings_as_errors = false

# For clang, add the following lines:
is_clang = true
clang_base_path = "/path/to/your/llvm_build"
clang_use_chrome_plugins = false
use_lld = false


# END PASTE Save file and it will generate build configuration

# ffmpeg fix (see https://docs.google.com/document/d/14bqZ9NISsyEO3948wehhJ7wc9deTIz-yHUhF1MQp7Po/edit#heading=h.t7yeo9by5dyr for info on build system)
cd third_party/ffmpeg
# OPTION 1 - Manually generate configuration
curl "https://wiki.raptorcs.com/w/images/7/7b/0001-Add-support-for-ppc64.patch" | patch -p1
./chromium/scripts/build_ffmpeg.py linux ppc64
./chromium/scripts/generate_gn.py
./chromium/scripts/copy_config.sh 
# END OPTION 1
# OPTION 2 - Apply pre-generated configuration
curl "https://wiki.raptorcs.com/w/images/2/20/0001-third_party-ffmpeg-Add-pre-generated-ppc64-configura.patch" | patch -p1
# END OPTION 2
cd ../../

# Build Chromium
ninja -C out/Default chrome

# Clean everything to do a rebuild
ninja -C out/Default -t clean
  • Notes about QT WebEngine by Sharkcz
    • patches mentioned here can be used for building QT WebEngine (5.11.2 from http://code.qt.io/qt/qtwebengine.git), chromium 69-based
    • had to set "use_jumbo_build = false" to workaround a build issue in V8, somehow it converted mul into mulld and div into divd, leading to errors like " kExprF32Mulld not defined"
    • have problems building ffmpeg, so the final link failed
      • seems the steps for ffmpeg are (because generate_gn reads the results of a build, needs clang in the default config => modified to use gcc, clang causes issues with missing symbols
        • build_ffmpeg.py linux ppc64
        • generate_gn.py
        • copy_config.sh
    • https://wiki.raptorcs.com/wiki/File:Qtwebengine.patch - for WebEngine itself
    • https://wiki.raptorcs.com/wiki/File:Webengine-chromium.patch - for the bundled Chromium (**Note:** this contains an outdated version of the 0005-sandbox-linux-seccomp-bpf-Add-ppc64-syscall-stub patch above which does not work -- you must use the updated one dated 21-Sep from above or you will get runtime linker failures)
      • rename third_party/ffmpeg/ffmpeg_generated.gni.ppc64 to ffmpeg_generated.gni - this is a preparation for packaging work, ideally we would need a complete ffmpeg_generated.gni regenerated for all arches
      • PushAllRegisters() needs a real implementation
    • how to use
      • apply the patchsets
      • qmake-qt5
      • make
      • wait a while and then enjoy :-)
    • https://src.fedoraproject.org/fork/sharkcz/rpms/qt5-qtwebengine/commits/ppc is my work in progress on the Fedora qt5-qtwebengine package
    • https://koji.fedoraproject.org/koji/taskinfo?taskID=29540352 is temporary location for the result
[sharkcz@guest05 lib]$ ll *5.11.2
-rwxrwxr-x. 1 sharkcz sharkcz 262536744 Sep  7 12:03 libQt5WebEngineCore.so.5.11.2
-rwxrwxr-x. 1 sharkcz sharkcz  11645504 Sep  7 12:04 libQt5WebEngine.so.5.11.2
-rwxrwxr-x. 1 sharkcz sharkcz   7154392 Sep  7 12:05 libQt5WebEngineWidgets.so.5.11.2

[sharkcz@guest05 lib]$ file libQt5WebEngineCore.so.5.11.2
libQt5WebEngineCore.so.5.11.2: ELF 64-bit LSB shared object, 64-bit PowerPC or cisco 7500, version 1 (SYSV), dynamically linked, BuildID[sha1]=39083e3d8053ec0ffaccfee7a8634d9d6954dbde, with debug_info, not stripped, too many notes (256)
Retrieved from "https://wiki.raptorcs.com/w/index.php?title=Porting/Chromium_(Old)&oldid=1935"