- 1 Background
- 2 Common Issues
- 2.1 Bootloader does not show up on monitor(s) attached to a discrete GPU
- 2.2 My AMD GPU works in petitboot but not the subsequent Linux OS
- 2.3 I want Petitboot via AST but the subsequent Linux OS console on a discrete GPU
- 2.4 Xorg will not start / crashes when a discrete GPU is installed
- 2.5 Monitor not detected in kernel 4.17+
- 2.6 Xorg crashes or is laggy with the AST VGA GPU
- 2.7 KDE is laggy with the AST VGA GPU
- 2.8 Display stuck at default low resolution with AST HDMI GPU
- 2.9 AMDGPU driver crashes after firmware update
- 3 Notes
Because OpenPOWER systems do not have a legacy graphics interface to fall back to, and as a result rely heavily on the running operating system and its drivers to handle display tasks, a few rough edges are exposed. This page attempts to document the current status of these rough edges and suggested workarounds pending actual fixes.
Bootloader does not show up on monitor(s) attached to a discrete GPU
Most modern discrete GPUs require firmware. As Talos™ II is aimed at a security-conscious audience, we do not currently include GPU firmware in the production firmware images. Instructions are available in the Users Guide to add firmware for your GPU to the PNOR if needed. Note that any added firmware may be able to access and modify data associated with the affected device(s); we strongly recommend you perform a security risk analysis before loading any firmware, and select open firmware where/if it is available.
If you are using a GPU that does not require firmware, or have already added any needed firmware files to the host PNOR, please ensure that the on-board VGA disable jumper (J10109) is capped. The bootloader output will preferentially show up on the on-board VGA port if it remains enabled.
Alternatively, you either use a serial console or VGA monitor / adapter to interact with the bootloader.
My AMD GPU works in petitboot but not the subsequent Linux OS
Older versions of the amdgpu driver (Linux 4.15 and below) have a bug where the connected outputs will not re-initialize after a kexec() while the driver is loaded. Kernel 4.16 and above does not appear to have this problem.
If you need to use kernel 4.15 or below, you can work around this issue by either:
- Enabling and using the VGA video output to access the bootloader (petitboot) -or-
- using a serial connection to control petitboot, and running the following commands prior to selecting an operating system via the petitboot menu:
echo 0 > /sys/class/vtconsole/vtcon1/bind rmmod amdgpu
I want Petitboot via AST but the subsequent Linux OS console on a discrete GPU
If you don't want to put GPU firmware in the PNOR but still want Linux tty on the discrete graphics, you'll find that you'll always get output on the AST first no matter what. Blacklisting the
ast module from loading is not sufficient on its own, you will need two kernel boot arguments:
For example on Ubuntu, this can be accomplished by changing
/etc/default/grub accordingly, like
and then running
Xorg will not start / crashes when a discrete GPU is installed
Installing more than one GPU into an OpenPOWER system (for instance, when adding a discrete GPU) exposes all GPUs directly to the operating system -- there is no concept of a "primary" GPU like there is on x86. Xorg does not handle this gracefully, tending to crash during autoconfiguration. At least one bug report has been filed but fixing the root cause of this issue (incorrect Xorg drivers binding to underlying DRM devices) does not seem to be an Xorg priority. Furthermore, Xorg does not properly handle domains during autoconfiguration per another bug report on a similar issue. Community effort in getting proper fixes into Xorg would be very useful, as the Xorg developers may want to see that more than one or two systems are impacted by these bugs before working on resolving them.
Two workarounds are available:
Workaround 1: Disable on-board VGA
Disable the on-board VGA output via the VGA disable jumper, J10109. See the Users Guide for additional information.
Workaround 2: Select desired GPU at runtime
The workaround to keep both devices active, or to retain the ability to switch in the active operating system, is fairly simple, and consists of explicitly assigning Xorg drivers for each installed GPU. For this example we'll show how to fix Xorg on Debian with an AMD WX7100 discrete GPU installed.
Step 1: Locate Bus Numbers
root@talos:~# lspci | grep VGA 0000:01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon Pro WX 7100] 0005:02:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 41)
Note the numbers to the left of the "VGA compatible controller" string. Each of these numbers is the PCI d:B:D.F[note 1] number of the GPU, and is unique to the slot(s) you have your GPU(s) installed in. As a result of this slot dependence, bus IDs may differ from those shown in this example; always use your bus IDs when following the steps below. This slot dependence means that if you move your GPU to a different slot you will need to update the bus ID associated with that GPU.
Step 2: Create Xorg Configuration Snippet
root@talos:~# mkdir /etc/X11/xorg.conf.d
Create and open
/etc/X11/xorg.conf.d/21-gpu-driver.conf for editing, then adjust the following template with your GPU information. Pay close attention to the BusID and Driver fields, as they must match your installed GPU(s). Note that Xorg uses decimal numbering, not hexadecimal like
lspci, so you will need to convert the numbers within the colons of the lspci output to decimal in order to constrict a valid Xorg BusID. Furthermore, xorg doesn't use leading zeroes like
lspci does; these must be stripped off when assembling the Xorg BusID. Finally, Xorg expects to see a BusID assembled as "PCI:B@d:D:F" (note Bus and Domain are swapped), and should not be assembled not using the format shown by
# AST2500 Section "Device" Identifier "GPU0" Driver "modesetting" BusID "PCI:2@5:0:0" VendorName "ASpeed Corporation" EndSection # WX7100 Section "Device" Identifier "GPU1" Driver "modesetting" # or amdgpu if you have xf86-video-amdgpu installed BusID "PCI:1@0:0:0" VendorName "AMD Corporation" EndSection # this is absolutely necessary, it tells xorg which GPU to use for the screen Section "Screen" Identifier "Screen0" Device "GPU1" EndSection
Save and exit the configuration snippet file, then restart Xorg. Your GPUs should now function as intended. If Xorg still does not start, make sure that the appropriate kernel driver (such as
amdgpu for the example above, keep in mind that the Xorg driver and the kernel driver are separate and distinct) has been loaded:
root@talos:~# modprobe amdgpu
You can use the generic modesetting Xorg driver for AMD GPUs, or you can use amdgpu from xf86-video-amdgpu. The generic modesetting driver has been reported to work perfectly fine on a Talos with various GPUs, so there is likely no practical reason to use the driver-specific DDX.
Step 3 (optional): Disable Integrated Video
To disable the ASpeed VGA in the booted OS completely, you can use the
modprobe.blacklist=ast approach on kernel command line, refer to the "I want Petitboot via AST but the subsequent Linux OS console on a discrete GPU" section above for more information. This method is universal/works on all distributions. The ASpeed VGA will still show up in
lspci afterwards, which is normal, as you haven't disabled the hardware, just the driver.
With this done, it should be possible to remove the device section in the X.Org configuration file for the onboard VGA, but you can also just leave it there if you want, regardless of whether the driver is loaded or not.
There are alternative ways to blacklist the
ast kernel driver. For example on Debian based systems, create a new file
/etc/modprobe.d/ast-blacklist.conf and place the following line inside the new file:
You may need to reboot if the
ast DRM driver has already loaded. Alternatively, you may try to unbind and unload the
ast driver as follows (assuming the
ast driver is bound to vtcon0):
root@talos:~# echo 0 > /sys/class/vtconsole/vtcon0/bind root@talos:~# rmmod ast
Monitor not detected in kernel 4.17+
Petitboot shows up fine, but there is no output for the host OS. It has been reported as bug 107049, the workaround is to append
amdgpu.dc=0 to the kernel parameters. This is often associated with a host dmesg trace of
[drm] Cannot find any crtc or sizes
EDIT 2019-05-02: Seems the
dc=0 workaround is not required any more, tested with 5.1-rc7
Xorg crashes or is laggy with the AST VGA GPU
Xorg seems to enable GLAMOR by default on many operating systems (such as Debian Buster). GLAMOR is a translation layer that converts 2D graphics operations to 3D graphics operations. This makes sense when 3D GPU acceleration is available, but when using a simple unaccelerated 2D GPU like the AST VGA GPU, the result is that 2D operations get converted to 3D operations by GLAMOR and are then converted back to 2D by llvmpipe, which introduces significant overhead.
In addition, on Debian Buster, GLAMOR has been observed to crash when used in conjunction with llvmpipe.
You can disable GLAMOR by saving the following text file as
Section "Device" Identifier "nogpu" Driver "modesetting" Option "Accelmethod" "none" EndSection Section "Module" Disable "glamoregl" EndSection
This bug was probably fixed by commit 1e3c5d614ee33d9eac1d2cf6366feeb8341fc0f4 in upstream Xorg; it is unknown how quickly the fix will make its way to GNU/Linux distributions.
KDE is laggy with the AST VGA GPU
KDE's default compositor uses OpenGL. This makes sense when 3D GPU acceleration is available, but when using a simple unaccelerated 2D GPU like the AST VGA GPU, the result is that 2D operations get converted to 3D operations by KDE's compositor and are then converted back to 2D by llvmpipe, which introduces significant overhead.
To fix this, go to
System Settings →
Display and Monitor →
Compositor, and select
XRender as the
Rendering backend. You'll probably also want to select
Smooth (slower) as the
Scale method (it's still much faster than OpenGL, and it looks quite a bit better).
Display stuck at default low resolution with AST HDMI GPU
As of 05/24/2019 upstream Linux kernels do not have driver support for the IT66121FN HDMI transceiver. This is being actively worked by Raptor Computing Systems and the larger ppc64el community. Until support is added, you will need to force the correct resolution in Xorg. The general process for discovering bus IDs etc. is detailed above in the AMD GPU section; you will need to extend the result with custom modelines as shown below:
# AST2500 Section "Device" Identifier "GPU0" Driver "modesetting" BusID "PCI:2@5:0:0" VendorName "ASpeed Corporation" EndSection # configure as appropriate for your monitor -- a standard 1080p screen is assumed below Section "Monitor" Identifier "Monitor0" HorizSync 30.0-70.0 VertRefresh 50.0-70.0 Modeline "1920x1080" 172.80 1920 2040 2248 2576 1080 1081 1084 1118 -HSync +Vsync EndSection # this is absolutely necessary, it tells xorg which GPU to use for the screen Section "Screen" Identifier "Screen0" Monitor "Monitor0" Device "GPU0" DefaultDepth 24 SubSection "Display" Depth 24 Modes "1920x1080" EndSubSection EndSection
AMDGPU driver crashes after firmware update
The GPU only allows loading one firmware after an ASIC reset, so the firmware used by the skiroot kernel and the host kernel must be the same. See FreeDesktop.org bug 108585 for more details.
- PCI Domain:Bus:Device.Function