Porting/Chromium (Old)

From RCS Wiki
Revision as of 14:57, 27 February 2019 by Shawnanastasio (talk | contribs) (change warning message)
Jump to navigation Jump to search

Further information on: Work specific to Big Endian support

Further information on: Electron support

For Debian-based systems, Raptor Engineering provides prebuilt binaries of the Chromium port, available here

Note: At the time of writing (2019-02-27), these binaries are out of date and do not include the latest security fixes from upstream or the latest versions of these patches. It is recommended to build from source by following the instructions below.

  • What to port to ppc
    • nacl (Not porting since it's been deprecated by Google.)

# Install Ubuntu 18.04 (Bionic Beaver) LTS ppc64le in VM with good amount of CPUs and ~16GB RAM, the more CPUs the more RAM, and around 200GB disk space

sudo apt update
sudo apt full-upgrade
sudo apt install git vim cmake python libcups2-dev pkg-config libnss3-dev libssl-dev libglib2.0-dev libgnome-keyring-dev libpango1.0-dev libdbus-1-dev libatk1.0-dev libatk-bridge2.0-dev libgtk-3-dev libkrb5-dev libpulse-dev libxss-dev re2c subversion curl libasound2-dev libpci-dev mesa-common-dev gperf bison nodejs uuid-dev clang-format

##begin alternate - use a xenial-chroot instead of a VM
#Create chroot
sudo debootstrap --include=software-properties-common xenial xenial-chroot http://us.ports.ubuntu.com/ubuntu-ports/ubuntu-ports/ 

#must be mapped for chroot to function properly
sudo mount -t proc none xenial-chroot/proc/
sudo mount -o bind /sys/ xenial-chroot/sys/
sudo mount -o bind /dev/shm/ xenial-chroot/dev/shm/

#enter chroot
sudo chroot xenial-chroot

#add additional apt sources
echo "deb http://us.ports.ubuntu.com/ubuntu-ports/ubuntu-ports xenial-updates main" >> /etc/apt/sources.list
echo "deb http://us.ports.ubuntu.com/ubuntu-ports/ubuntu-ports xenial universe" >> /etc/apt/sources.list
echo "deb http://us.ports.ubuntu.com/ubuntu-ports/ubuntu-ports xenial-updates universe" >> /etc/apt/sources.list

#update and add software as mentioned above
apt update
apt full-upgrade
apt install <packages listed above>

#get nodejs and libatspi2.0-dev from bionic:
echo "deb http://us.ports.ubuntu.com/ubuntu-ports/ubuntu-ports bionic main universe" >> /etc/apt/sources.list
apt update
apt-get install -t bionic nodejs libatspi2.0-dev

#add chroot user
useradd -m build-user -s /bin/bash
su build-user
echo 'cd' >> ~/.bashrc
cd
##end alternate - use a xenial-chroot instead of a VM


cd
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git

echo 'export PATH="$PATH:${HOME}/ninja"' >> ~/.bashrc
echo 'export PATH="$PATH:${HOME}/depot_tools"' >> ~/.bashrc
echo 'export VPYTHON_BYPASS="manually managed python not supported by chrome operations"' >> ~/.bashrc
. ~/.bashrc

git clone git://github.com/ninja-build/ninja.git
cd ninja
git checkout release
./configure.py --bootstrap
which ninja # ensure it is right ninja bin, else figure out a way so it is the right one that comes up


cd
git clone https://gn.googlesource.com/gn

export CC=gcc
export CXX=g++
export AR=ar

cd gn
python build/gen.py --no-sysroot
cat out/build.ninja | sed s/-stdlib=libstdc++//g | tee out/build.ninja
ninja -C out

unset CXX
unset CC
unset AR

# Setup clang (HIGHLY RECOMMENDED)
# Building chromium with gcc is an uphill battle as the devs only test with clang.
# It is possible to set up a local clang compiler with the exact revision chromium expects by doing the following:

# Check out LLVM (rev 353250)
svn checkout --force https://llvm.org/svn/llvm-project/llvm/trunk@353250 llvm

# Check out Clang (rev 353250)
svn checkout --force https://llvm.org/svn/llvm-project/cfe/trunk@353250 llvm/tools/clang

# Check out compiler-rt (rev 353250)
svn checkout --force https://llvm.org/svn/llvm-project/compiler-rt/trunk@353250 llvm/compiler-rt

# Make build directory and build
mkdir llvm_build
cd llvm_build
cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD="PowerPC" -G "Unix Makefiles" ../llvm
make -j64 # Replace with no. of threads in your box

# Work around libatomic problems under Clang
# See https://github.com/cquery-project/cquery/issues/382 for a related problem in a different project
cd /usr/lib/powerpc64le-linux-gnu
ln -s libatomic.so.1 libatomic.so

# Setup and build chromium tree
cd
mkdir chromium
cd chromium

# The following command will fail with a message "failed to resolve package version".
# Once it does, simply continue following the instructions. The subsequent patches will resolve the issue.
fetch --no-history --nohooks chromium

curl "https://wiki.raptorcs.com/w/images/4/45/Src-DEPS.patch" | patch -p1 src/DEPS # If this patchfile stops working, ignore the error, see "Patchfiles uploaded"
curl "https://wiki.raptorcs.com/w/images/1/10/Binutils-download.py-PPC.patch" | patch -p1 src/third_party/binutils/download.py

# Disable nacl in gclient configuration
sed -i 's/\"custom_vars\"\: {},/\"custom_vars\"\: { \"checkout_nacl\"\: False },/g' .gclient

gclient sync
gclient runhooks

mv src/buildtools/linux64/gn src/buildtools/linux64/gn.old
ln -s $HOME/gn/out/gn src/buildtools/linux64/gn

cd src

# build system patches
curl "https://wiki.raptorcs.com/w/images/6/69/Buildtools-third_party-libc%2B%2B-trunk-include-thread-PPC.patch" | patch -p1 buildtools/third_party/libc++/trunk/include/thread
curl "https://wiki.raptorcs.com/w/images/5/5c/0006-build-toolchain-allow-clang-on-ppc64.patch"                  | patch -p1

# breakpad fix
cd third_party/breakpad/breakpad
curl "https://wiki.raptorcs.com/w/images/f/f7/0001-Implement-ppc64-support-on-linux.patch"                      | patch -p1
cd ../../..

# crashpad fix
cd third_party/crashpad/crashpad
curl "https://wiki.raptorcs.com/w/images/8/88/0001-Implement-support-for-PPC64-on-Linux.patch"                  | patch -p1
cd ../../..
curl "https://wiki.raptorcs.com/w/images/6/6e/0002-Include-cstddef-to-fix-build.patch"                          | patch -p1

# webrtc patches
curl "https://wiki.raptorcs.com/w/images/b/b2/Rtc_base-system-arch.h-PPC.patch" | patch -p1 third_party/webrtc/rtc_base/system/arch.h
curl "https://wiki.raptorcs.com/w/images/9/96/Modules-desktop_capture-differ_block.cc-PPC.patch" | patch -p1 third_party/webrtc/modules/desktop_capture/differ_block.cc

# sandbox patches
curl "https://wiki.raptorcs.com/w/images/a/ab/0001-sandbox-linux-bpf_dsl-Update-syscall-ranges-for-ppc6.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/3/30/0002-sandbox-linux-bpf_dsl-Modify-seccomp_macros-to-add-s.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/6/60/0003-sandbox-linux-system_headers-Update-linux-seccomp-he.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/e/ec/0004-sandbox-linux-system_headers-Update-linux-signal-hea.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/b/b3/0005-sandbox-linux-seccomp-bpf-Add-ppc64-syscall-stub.patch"     | patch -p1
curl "https://wiki.raptorcs.com/w/images/f/f5/Sandbox-linux-services-credentials.cc-PPC.patch"                 | patch -p1
curl "https://wiki.raptorcs.com/w/images/b/be/0001-sandbox-Enable-seccomp_bpf-for-ppc64.patch"                 | patch -p1
curl "https://wiki.raptorcs.com/w/images/3/38/0001-sandbox-linux-Implement-partial-support-for-ppc64-sy.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/b/bb/0001-sandbox-linux-seccomp-bpf-helpers-Fix-TCGETS-declara.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/e/e7/0001-sandbox-linux-seccomp-bpf-helpers-Skip-vserver-sysca.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/4/43/0001-sandbox-linux-Update-syscall-helpers-lists-for-ppc64.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/5/5a/0001-sandbox-linux-Update-IsSyscallAllowed-in-broker_proc.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/0/0d/0001-linux-seccomp-bpf-ppc64-glibc-workaround-in-SIGSYS-h.patch" | patch -p1

# service_manager patches
curl "https://wiki.raptorcs.com/w/images/c/c9/0001-services-service_manager-sandbox-linux-Fix-TCGETS-de.patch" | patch -p1

# third_party patches
curl "https://wiki.raptorcs.com/w/images/8/8a/0001-third_party-angle-Include-missing-header-cstddef-in-.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/f/fc/0001-third_party-lss-Don-t-look-for-mmap2-on-ppc64.patch"        | patch -p1
curl "https://wiki.raptorcs.com/w/images/8/8d/0001-DONTMERGE-third_party-node-Use-system-nodejs-binary.patch"  | patch -p1
curl "https://wiki.raptorcs.com/w/images/0/0b/0002-third_party-closure_compiler-Remove-useless-JVM-flag.patch" | patch -p1
curl "https://wiki.raptorcs.com/w/images/5/5f/0001-third_party-blink-Fix-build-on-ppc64.patch"                 | patch -p1
curl "https://wiki.raptorcs.com/w/images/8/84/0001-components-update_client-Fix-building-on-ppc64.patch"       | patch -p1
curl "https://wiki.raptorcs.com/w/images/0/02/0001-Reenable-VSX-in-libpng.patch"                               | patch -p1
curl "https://wiki.raptorcs.com/w/images/2/28/0001-third_party-boringssl-Properly-detect-ppc64le-in-BUI.patch" | patch -p1

# page size fixes
curl "https://wiki.raptorcs.com/w/images/f/f3/0001-base-allocator-Use-64k-page-sizes-on-ppc64.patch"           | patch -p1
curl "https://wiki.raptorcs.com/w/images/5/5c/0001-Apply-blink-64k-page-workaround-to-ppc64-systems.patch"     | patch -p1

# libvpx fix
curl "https://wiki.raptorcs.com/w/images/8/80/0001-third_party-libvpx-Properly-generate-gni-on-ppc64.patch"    | patch -p1
# VP9 rendering is broken with the ppc64 VSX code in libvpx. You can optionally apply the following patch to work around it:
curl "https://wiki.raptorcs.com/w/images/5/5e/HACK-third_party-libvpx-use-generic-gnu.patch"                   | patch -p1

cd third_party/libvpx
mkdir source/config/linux/ppc64
./generate_gni.sh
cd ../../

# pdfium fixes
cd third_party/pdfium
# The following patch is no longer needed on recent chromium revisions and will fail to apply
#curl "https://wiki.raptorcs.com/w/images/4/41/0001-Add-spin-yield-support-for-ppc64.patch"                      | patch -p1

curl "https://wiki.raptorcs.com/w/images/2/25/0001-pdfium-allocator-Use-64k-page-sizes-on-ppc64.patch"          | patch -p1
cd ../..

# v8 fixes
cd v8
curl "https://wiki.raptorcs.com/w/images/3/3c/0001-DONTMERGE-Disable-v8-unit-tests.patch"                       | patch -p1
curl "https://wiki.raptorcs.com/w/images/b/b0/0001-Force-baseline-POWER8-AltiVec-VSX-CPU-features-when-.patch"  | patch -p1
# The following two patches are upstreamed and no longer apply
#curl "https://wiki.raptorcs.com/w/images/9/9c/0001-PPC-s390-iwyu-Remove-include-heap-inl.h-counters.h.patch"    | patch -p1
#curl "https://wiki.raptorcs.com/w/images/c/c2/0001-Fix-build-issues-on-64k-systems.patch"                       | patch -p1
cd ..

# database fix
curl "https://wiki.raptorcs.com/w/images/d/df/0001-Properly-detect-little-endian-PPC64-systems.patch"           | patch -p1

# blink fix
curl "https://wiki.raptorcs.com/w/images/a/aa/0001-third_party-blink-Add-SaveRegisters-implementation-w.patch"  | patch -p1

### GCC ONLY! (NO LONGER NEEDED, MERGED UPSTREAM)

# quick fix https://bugs.chromium.org/p/chromium/issues/detail?id=882347&can=2&q=gcc&colspec=ID%20Pri%20M%20Stars%20ReleaseBlock%20Component%20Status%20Owner%20Summary%20OS%20Modified
#git checkout 7b4f22bd25461cb32236c07699b029c7411ba494 -- net/third_party/quic/core/frames/quic_frame.h

# blink gcc fix https://bugs.chromium.org/p/chromium/issues/detail?id=882161#
#curl "https://wiki.raptorcs.com/w/images/d/de/Chromium-blink-snapcontainerdata.patch" | patch -p1

### END GCC ONLY

gn args out/Default

# PASTE IN EDITOR

# Release mode
is_component_build = false
is_debug = false

# Disable broken features
enable_nacl = false
treat_warnings_as_errors = false

# For clang, add the following lines:
is_clang = true
clang_base_path = "/path/to/your/llvm_build"
clang_use_chrome_plugins = false
use_lld = false


# END PASTE Save file and it will generate build configuration

# ffmpeg fix (see https://docs.google.com/document/d/14bqZ9NISsyEO3948wehhJ7wc9deTIz-yHUhF1MQp7Po/edit#heading=h.t7yeo9by5dyr for info on build system)
cd third_party/ffmpeg
# OPTION 1 - Manually generate configuration
curl "https://wiki.raptorcs.com/w/images/7/7b/0001-Add-support-for-ppc64.patch" | patch -p1
./chromium/scripts/build_ffmpeg.py linux ppc64
./chromium/scripts/generate_gn.py
./chromium/scripts/copy_config.sh 
# END OPTION 1
# OPTION 2 - Apply pre-generated configuration
curl "https://wiki.raptorcs.com/w/images/2/20/0001-third_party-ffmpeg-Add-pre-generated-ppc64-configura.patch" | patch -p1
# END OPTION 2
cd ../../

# Build Chromium
ninja -C out/Default chrome

# Clean everything to do a rebuild
ninja -C out/Default -t clean
  • Notes about QT WebEngine by Sharkcz
    • patches mentioned here can be used for building QT WebEngine (5.11.2 from http://code.qt.io/qt/qtwebengine.git), chromium 69-based
    • had to set "use_jumbo_build = false" to workaround a build issue in V8, somehow it converted mul into mulld and div into divd, leading to errors like " kExprF32Mulld not defined"
    • have problems building ffmpeg, so the final link failed
      • seems the steps for ffmpeg are (because generate_gn reads the results of a build, needs clang in the default config => modified to use gcc, clang causes issues with missing symbols
        • build_ffmpeg.py linux ppc64
        • generate_gn.py
        • copy_config.sh
    • https://wiki.raptorcs.com/wiki/File:Qtwebengine.patch - for WebEngine itself
    • https://wiki.raptorcs.com/wiki/File:Webengine-chromium.patch - for the bundled Chromium (**Note:** this contains an outdated version of the 0005-sandbox-linux-seccomp-bpf-Add-ppc64-syscall-stub patch above which does not work -- you must use the updated one dated 21-Sep from above or you will get runtime linker failures)
      • rename third_party/ffmpeg/ffmpeg_generated.gni.ppc64 to ffmpeg_generated.gni - this is a preparation for packaging work, ideally we would need a complete ffmpeg_generated.gni regenerated for all arches
      • PushAllRegisters() needs a real implementation
    • how to use
      • apply the patchsets
      • qmake-qt5
      • make
      • wait a while and then enjoy :-)
    • https://src.fedoraproject.org/fork/sharkcz/rpms/qt5-qtwebengine/commits/ppc is my work in progress on the Fedora qt5-qtwebengine package
    • https://koji.fedoraproject.org/koji/taskinfo?taskID=29540352 is temporary location for the result
[sharkcz@guest05 lib]$ ll *5.11.2
-rwxrwxr-x. 1 sharkcz sharkcz 262536744 Sep  7 12:03 libQt5WebEngineCore.so.5.11.2
-rwxrwxr-x. 1 sharkcz sharkcz  11645504 Sep  7 12:04 libQt5WebEngine.so.5.11.2
-rwxrwxr-x. 1 sharkcz sharkcz   7154392 Sep  7 12:05 libQt5WebEngineWidgets.so.5.11.2

[sharkcz@guest05 lib]$ file libQt5WebEngineCore.so.5.11.2
libQt5WebEngineCore.so.5.11.2: ELF 64-bit LSB shared object, 64-bit PowerPC or cisco 7500, version 1 (SYSV), dynamically linked, BuildID[sha1]=39083e3d8053ec0ffaccfee7a8634d9d6954dbde, with debug_info, not stripped, too many notes (256)
Retrieved from "https://wiki.raptorcs.com/w/index.php?title=Porting/Chromium_(Old)&oldid=1980"