Porting/Chromium/BE

From RCS Wiki
Revision as of 07:02, 18 May 2019 by MarcusC (talk | contribs)
Jump to navigation Jump to search

Changes required to Chromium to work on big endian

  • Skia
- Chromium assumes that the pixel format, when viewed as a 32-bit word, is either ARGB or ABGR. However, third_party/skia/include/core/SkImageInfo.h contains a check that the pixel format, when viewed as four 8-bit bytes, is either BGRA or RGBA. Hence either this check needs to be relaxed (Maybe it is bogus in the first place? The comments talk about "native ARGB 32-bit", yet the test goes out of its way to test the ordering as bytes and not as 32-bit words), or Chromium needs to learn about a different pixel format. Selected BGRA component order, which works in Skia and seems to be workable in Chromium as well.
  • Datapack
    • ui/base/resource/data_pack.cc
    - Need to byteswap metadata on input (resource file format is specified to use LE) Done
    • chrome/browser/themes/browser_theme_pack.cc
    - "datapack assumes little endian" Done
  • Gfx
    • ui/gfx/codec/jpeg_codec.cc
    - Selection of out_color_space when format is FORMAT_SkBitmap needs to check component order of SkPMColor as bytes, not as shift Done
    • ui/gfx/color_analysis.cc
    - Functions accessing SkColor format pixels as bytes need updating for big endian. Done
    • ui/gfx/codec/png_codec.h and ui/gfx/codec/png_codec.cc
    - A new FORMAT_ARGB is needed for decoding so that color_analysis can decode into in-memory SkColor on big endian. Done
  • Net
    • net/cert/crl_set.cc
    - "assumes little endian" Done
    • components/safe_browsing/db/v4_rice.cc
    - #error The code below assumes little-endianness. Done
  • Media
    • media/ffmpeg/ffmpeg_common.cc and media/formats/mp4/box_definitions.cc
    - #error The code below assumes little-endianness. Removed check
    • media/renderers/paint_canvas_video_renderer.cc
    - Depending on choice of Skia pixel format, a new LIBYUV mapping set may be needed (see also third_party/libyuv below). Adjusted check
  • I18N
    • base/i18n/icu_util.cc
    - Needs to use "icudtb.dat" instead of "icudtl.dat" on BE Done
  • Sandbox
    • sandbox/linux/system_headers/linux_seccomp.h
    - The definition of the big endian variant is incorrect (contains a bogus "__AUDIO_ARCH_BE"). Also, it would be better to define two different macros AUDIT_ARCH_PPC64 and AUDIT_ARCH_PPC64LE (because that is how it actually looks in <linux/audit.h>), and choose the correct one in sandbox/linux/bpf_dsl/seccomp_macros.h instead. Done
    • sandbox/linux/bpf_dsl/seccomp_macros.h
    - The definitions of the macros SECCOMP_IP_MSB_IDX, SECCOMP_IP_LSB_IDX, SECCOMP_ARG_MSB_IDX, and SECCOMP_ARG_LSB_IDX need to take endianness into account Done
  • Partition allocator
    • base/allocator/partition_allocator/partition_freelist_entry.h
    - A different pointer transformation function is used on BE compared to on LE. This is not an issue in itself, but it exposes a bug that NULL pointers are sometimes not transformed when they ought to be (the LE transformation function is unity for NULL). Done
  • Pickle
    • base/pickle.cc
    - ReadBool accesses serialized bools as "bool" (8 bit) despite WriteBool serializing them as "int" (32 bit) Done
  • SHA1
    • base/sha1.cc
    - Contains some byteswaps which should only happen on little endian Done
  • Url
    • url/url_canon_host.cc
    - Uses a reinterpret_cast to get an unsigned char pointer into an unsigned int, for no good reason. Done
  • Aura
    • ui/aura/mus/os_exchange_data_provider_mus.cc
    - Outputs UTF-16 in native byte order (which is probably correct), but hardcodes an LE BOM. Done
  • Unit tests
    • base/memory/shared_memory_mapping_unittest.cc, base/metrics/bucket_ranges_unittest.cc, and base/strings/string_number_conversions_unittest.cc
    - These unit tests expose endianness in one way or another and need to be adjusted to run correctly on BE Done
    • ui/gfx/skbitmap_operations_unittest.cc
    - The CreateHSLShiftedBitmapHueOnly test uses the wrong function to set SkPMColor values, which breaks on BE (and should break on Android as well..) Done
    • skia/ext/image_operation_unittest.cc and ui/gfx/codec/png_codec_unittest.cc
    - There is code here which needs to convert between SkColor and SkPMColor (which can be different, even on LE) when accessing pixels directly. Done
    • chrome/test/chromedriver/chrome_launcher.cc and chrome/test/chromedriver/util.cc
    - Missing byteswaps when parsing zip and crx headers Done
  • third_party/skia
    • include/private/GrTypesPriv.h
    - #error "Skia gpu currently assumes little endian" Removed check
    • src/core/SkPixmap.cpp
    - getColor() needs to use something other than SkSwizzle_BGRA_to_PMColor to convert SkPMColor to SkColor, because SkColor is defined by fixed 32-bit shifts, while "BGRA" is defined by 8-bit component order in memory. Done
    • src/core/SkUnPreMultiplyPriv.h and src/images/SkImageEncoderFns.h
    - The scanline transformation functions need to take endianness into account since the pixel formats are defined by memory order, but the conversion functions use 32-bit loads and hardcoded shifts. Done
    • src/opts/Sk4px_none.h
    - Sk4px::alphas, Sk4px::zeroAlphas, Sk4px::zeroColors: "This method assumes little-endian." Removed check
    • src/opts/SkXfermode_opts.h
    - a_rgb assumes specific component order Removed check
    • src/opts/SkBlitRow_opts.h
    - The "portable" implementation of blit_row_s32a_opaque hardcodes alpha bit position Done
    • src/opts/SkRasterPipeline_opts.h
    - Pixel load and store ops need to take endianness into account Done
    • src/utils/SkJSON.cpp and src/utils/SkJSON.h
    - Tagged value implementation needs to be adapted for BE Done
    • src/codec/SkCodecPriv.h
    - Missing include of "SkEndian.h" Done
    • third_party/skcsm/skcms.cc
    - The read_big_* functions byteswap the values read even on BE Done Also eval_curve does this. Done
    • third_party/skcms/src/Transform_inl.h
    - Pixel load and store ops need to take endianness into account Done
  • third_party/boringssl/src
    • include/openssl/base.h
    - Need to recognize BE ppc64 Done
    • crypto/fipsmodule/bn/bytes.c
    - Bignum I/O functions need adaption for BE Done
    • crypto/fipsmodule/ecdsa/ecdsa.c and third_party/fiat/p256.c
    - More bignum stuff needing adaption for BE Done
    • crypto/asn1/a_int.c, crypto/fipsmodule/modes/gcm.c, crypto/fipsmodule/modes/internal.h, and crypto/fipsmodule/rand/ctrdrbg.c
    - Hardcoded swaps which need to be removed on BE Done
    • crypto/fipsmodule/modes/polyval.c
    - reverse_and_mulX_ghash does both byte-swapping and arithmetic. The order needs to be reversed on BE. Done
    • crypto/poly1305/poly1305.c
    - Defines specific LE access functions, but does not do any actual serialization or swapping in them. Done
    • crypto/curve25519/spake25519.c and third_party/fiat/curve25519.c
    - The arithmetic operations on LE data (addition, comparison) need swaps on BE. Done
    • crypto/evp/scrypt.c
    - Need to swap the input and and output blocks on BE. Done
    • crypto/chacha/chacha.c
    - LE access functions are defined, and correct, but are only used in one of the two CHACHA variants. Done
    • crypto/cipher_extra/e_aesgcmsiv.c
    - Counter and length fields need swapping on BE. Done
    • crypto/compiler_test.cc
    - There is a test which deliberately fails if the host is not LE... Done
  • third_party/blink
    • renderer/platform/graphics/gpu/webgl_image_conversion.cc
    - Does contain BE support, but a variable rename refactoring is missing from the BE code Done
    • renderer/platform/graphics/logging_canvas.cc
    - Does contain BE support, but a variable rename refactoring is missing from the BE code Done
    • renderer/platform/image-decoders/jpeg/jpeg_image_decoder.cc
    - #error Blink assumes a little-endian target. Done
    • renderer/platform/image-decoders/webp/webp_image_decoder.cc
    - #error Blink assumes a little-endian target. Done
    • renderer/platform/heap/asm/SaveRegisters_ppc64.S
    - Needs to handle ELF ABI v1 Done
    • renderer/platform/image-decoders/bmp/bmp_image_reader.h
    - Needs to byteswap BMP header fields on BE. Done
    • renderer/platform/graphics/graphics_context_test.cc
    - Should use SkGetPackedA32() instead of hardcoding >>24. Done
    - Should use SkPackARGB32NoCheck() instead of hardcoding 32-bit pixel values. Done
    • renderer/platform/graphics/gpu/webgl_image_conversion_test.cc
    - The BE version of a test vector has a misspelled variable name. Done
    • renderer/platform/graphics/highcontrast/highcontrast_classifier.cc
    - The TensorFlow floating point parameters are initialized from a series of bytes. Should be initialized from floating point values instead to remove endianness (and floating point format) dependencies. Done
  • third_party/flatbuffers/src
    • include/flatbuffers/base.h
    - The ByteSwap code does not work on C++11 due to the default constructor of the unions getting deleted Done
    • include/flatbuffers/minireflect.h
    - IterateValue dereferences reinterpret_cast:ed pointers into LE data Done
  • third_party/perfetto
    • include/perfetto/base/utils.h
    - Page size hardcoded as 4096
    - "AssumeLittleEndian" template asserts even though never instantiated Done
    • src/protozero/message.cc
    - float and double handling need adjusting for BE (according to comment) Done
    • src/protozero/proto_decoder.cc
    - Byteswap implementations missing Done
  • third_party/ffmpeg
    • chromium/scripts/build_ffmpeg.py
    - Needs to recognize 'ppc64' as well as 'ppc64le' Done
  • third_party/webrtc
    • common_audio/wav_file.cc
    - WavReader::ReadSamples and WavWriter::WriteSamples need to byteswap the samples Done
    • common_audio/wav_header.cc
    - Conversion functions missing Done
  • third_party/modp_b64
    • BUILD.gn
    - modp_b64 actually supports BE, but in converting to their own build system Google managed to lose the check to define WORDS_BIGENDIAN Done
    • modp_b64.cc
    - The prototype of modp_b64_decode needs to be updated to use size_t instead of int for the lengths, as it has been done for the LE variant Done
  • third_party/crc32c
    • BUILD.gn
    - Needs to define BYTE_ORDER_BIG_ENDIAN=1 on BE Done
  • third_party/leveldatabase
    • port/port_chromium.h
    - Needs to set kLittleEndian depending on endianness Done
  • third_party/libvpx
- The build system should move VSX-specific sources into a separate source set, and set the cflags "-maltivec -mvsx" for it. In the original build system the cflags were set based on a glob of the source filename, but in Googles build system it has to be done manually. Done
  • third_party/libyuv
- Depending on choice of Skia pixel format, additional conversion function variants may be needed
  • base/third_party/dmg_fp
    • dtoa.cc
    - Google has hardcoded little endian mode Done

Failing testcases

Testsuite Testcase Result
aura_unittests OSExchangeDataProviderMusTest.TestHTML OK
base_unittests OutOfMemoryDeathTest.SecurityValloc Fail
(Also on LE)
PartitionAllocTest.DumpMemoryStats Crash
(Also on LE)
PartitionAllocTest.GenericAllocGetSize Fail
(Also on LE)
PartitionAllocTest.PurgeDiscardable Crash
(Also on LE)
PartitionAllocTest.Realloc Fail
(Also on LE)
StackTraceTest.TraceStackFramePointers Fail
(Also on LE)
StringUtilTest.FormatBytesUnlocalized OK
(Locale dep)
SysInfoTest.GetHardwareInfo Fail
(Also on LE)
TCMallocFreeTest.BadPointerInFirstPageOfTheLargeObject Fail
(Also on LE)
TCMallocFreeTest.DoubleFreeSmallObject Fail
(Also on LE)
blink_platform_unittests AffineTransformTest.ToString OK
(Locale dep)
BMPImageDecoderTest.isSizeAvailable OK
BMPImageDecoderTest.mergeBuffer OK
BMPImageDecoderTest.parseAndDecode OK
BitmapImageTest.APNGDecoder19 OK
BitmapImageTest.GifDecoderFrame0 OK
BitmapImageTest.GifDecoderFrame1 OK
BitmapImageTest.GifDecoderFrame3 OK
BitmapImageTest.GifDecoderMultiThreaded OK
BitmapImageTest.jpegHasColorProfile OK
BitmapImageTest.pngHasColorProfile OK
BitmapImageTest.webpHasColorProfile OK
DecodedImageTypeHistogramTest/DecodedImageTypeHistogramTest.ImageType/4 OK
DecodedImageTypeHistogramTest/DecodedImageTypeHistogramTest.ImageType/5 OK
DeferredImageDecoderTestWoPlatform.mixImagesBmp OK
DeferredImageDecoderTestWoPlatform.mixImagesIco OK
DoubleRectTest.ToString OK
(Locale dep)
DragImageTest.InterpolationNone OK
FloatRectTest.ToString OK
(Locale dep)
FontDescriptionTest.ToString OK
(Locale dep)
FontSelectionTypesTest.RequestToString OK
(Locale dep)
FontSelectionTypesTest.ValueToString OK
(Locale dep)
GraphicsContextHighConstrastTest OK
GraphicsContextTest.Recording OK
GraphicsContextTest.UnboundedDrawsAreClipped OK
HighContrastImageClassifierTest OK
ICOImageDecoderTests OK
ImageFrameTest.BlendRGBAPremultipliedF16Buffer OK
ImageFrameTest.BlendRGBARawF16Buffer OK
StaticPNGTests.DecodeHighBitDepthPngToHalfFloat OK
StaticWebPTests.isSizeAvailable OK
TransformationMatrixTest.ToString OK
(Locale dep)
chromedriver_unittests ProcessExtensions OK
UnzipSoleFile.Entry OK
components_unittests ActivationStateComputingThrottleSubFrameTest.Activate/0 Fail
(Also on LE)
BidirectionalStreamDelayRequestHeadersUntilFlush Fail
CastFramerTest.TestMessageFramerCompleteMessage Crash
CastFramerTest.TestUnparsableBodyProto Fail
CastTransportTest Crash
ChromeCTPolicyEnforcerTest Fail
(Also on LE)
DataAccessorTest.Mix Fail
DataReductionProxyInterceptorEndToEndTest.RedirectChainToHttps Crash
(Also on LE)
DocumentProviderTest.GenerateLastModifiedString Fail
(Also on LE)
ExpireHistoryTest Crash
FaviconHandlerTest.RedownloadExpiredPageUrlFavicon Fail
HistoryBackendTest Fail
ImageManagerTest.GetImageForURLNetworkCacheHit Timeout
ImageManagerTest.QueueImageRequest Timeout
MockCastSocketTest Crash
MultilingualSpellCheckTest Fail
NetExportFileWriterTest.AddEventCustomPath Fail
PasswordHashDataTest.CalculatePasswordHash Fail
PwgRasterTest.Encode Fail
QuarantineTest.FileCanBeAnnotatedWithNoGUID Fail
QuarantineTest.FileCanBeOpenedForReadAfterAnnotation Fail
RemoteSuggestionsProviderImplTest Fail
SelectFaviconFramesTest Fail
SenderPipeTest.TimeoutNoRead Fail
(Also on LE)
SenderPipeTest.TimeoutSmallRead Fail
(Also on LE)
SpellCheckTest Fail
SslCastSocketTest.TestConnectEndToEndWithRealSSL Fail
SubresourceFilterVerifiedRulesetDealerTest.OpenAndSetRulesetFileValidNoChecksum Fail
content_unittests BackgroundFetchDataManagerTest.GetInitializationData Timeout
BackgroundFetchDataManagerTest.UpdateRegistrationUI Timeout
BackgroundFetchImageHelpers.SerializeRoundTrip Timeout
CanvasCaptureHandlerTest.VerifyFrame Fail
DOMStorageDatabaseTest.TestCanOpenAndReadWebCoreDatabase Fail
PageStateSerializationTest Fail
dbus_unittests EndToEndAsyncTest.InvalidServiceName Crash
MessageTest.SetInvalidHeaders Fail
device_unittests OneWriterSeqLockTest.ManyThreads Fail
UsbDescriptorsTest.OddLengthStringDescriptor Fail
UsbDescriptorsTest.ReadStringDescriptors Fail
UsbDescriptorsTest.StringDescriptor Fail
gfx_unittests ColorAnalysisTest OK
ICCProfile OK
ImageTest.MultiResolutionImageSkiaToPNG OK
ImageTest.PNGEncodeFromPlatformDecodeToSkia OK
ImageUtilTest.JPEGEncodeAndDecode OK
NineImagePainterTest OK
PNGCodec OK
SimpleColorSpace OK
SkBitmapOperationsTest.CreateHSLShiftedBitmapHueOnly OK
SkBitmapOperationsTest.RotateImage OK
VectorIconTest.CorrectSizePainted OK
mojo_unittests ValidationTest.InputParser OK
(Locale dep)
skia_unittests ImageOperations.ResizeShouldAverageColors OK
ImageOperations.ScaleUp OK
ui_base_unittests OSExchangeDataTest.TestHTML Fail
unit_tests BackFwdMenuModelTest.FaviconLoadTest OK
BrowserThemePackTest.HiDpiThemeTest OK
BrowserViewTest.AccessibleWindowTitle Fail
(Also on LE)
CRLSetComponentInstallerTest OK
CaptureVisiblePageTest.URLsCapturableWithEitherActiveTabOrAllURLs Fail
(Also on LE)
ChromeAppIconTest.IconLifeCycle Fail
ChromeBrowserMainExtraPartsMetricsTest.VerifyTouchEventsEnabledIsNotRecordedAfterPostBrowserStart Fail
(Also on LE)
ChromePaths.UserCacheDir Fail
DownloadItemModelTest.InterruptTooltip Fail
(Also on LE)
DownloadItemModelTest.InterruptedStatus Fail
(Also on LE)
DownloadProtectionServiceTest.CheckClientDownloadSampledFile Crash
(Also on LE)
EnterpriseHardwarePlatformAPITest.GetHardwarePlatformInfoAllowed Fail
(Also on LE)
ExtensionIconManagerTest.ScaleFactors OK
ExtensionInfoGeneratorUnitTest.RuntimeHostPermissionsWithoutFeature Fail
(Also on LE)
MediaEngagementScoreTest.OverrideFieldTrial Fail
MurmurHash3UtilTest.MurmurHash3String Fail
PermissionMessageCombinationsUnittest Fail
(Also on LE)
PermissionsTest.GetWarningMessages_DeclarativeWebRequest Fail
(Also on LE)
PermissionsTest.GetWarningMessages_Socket_AnyHost Fail
(Also on LE)
SessionMonitorTest Crash
TabScorePredictorTest Fail
ThemeColorMatchesManifestTest.ThemeColor Crash
TwoPhaseUploaderTest Fail
WebAppIconGeneratorTest.ConstrainBitmapsToSizes OK
WebRtcRtpDumpWriterTest Fail
WifiStatusMonitorTest.QueryStatusAndRecordResponse Crash
webkit_unit_tests All/ScrollingCoordinatorTest.setupScrollbarLayerShouldNotCrash Crash
AllFields/FontBuilderAdditiveTest.OnlySetValueIsModified/7 Fail
CSSFontFaceSourceTest.HashCollision Fail
CanvasAsyncBlobCreatorTest.ColorManagedConvertToBlob Fail
CanvasRenderingContext2DTest.ColorManagedPutImageDataOnSRGBCanvas Fail
FrameSerializerTest.BlankFrames Crash
FrameSerializerTest.CSS Crash
ImageBitmapTest.ImageBitmapColorSpaceConversionImageData Fail
ImageDataTest.TestConvertPixelsFromCanvasPixelFormatToImageDataStorageFormat Fail
ImageDataTest.TestCreateImageDataFromStaticBitmapImage Fail
LazyFrameLoading/LazyLoadFramesTest.AboutBlankChildFrameNavigation Crash
ListenerLeakTest Crash
V8ScriptValueSerializerTest Fail
WebFrameSerializerSanitizationTest.RemoveIframeInHead Crash
WebFrameSerializerSanitizationTest.RemoveInlineScriptInAttributes Crash
WebFrameTest Crash
WebViewTest Crash
WindowProxyTest.ReinitializedAfterNavigation Crash

Patchfiles uploaded

Retrieved from "https://wiki.raptorcs.com/w/index.php?title=Porting/Chromium/BE&oldid=2466"