Difference between revisions of "From QEMU to BigEndian FreeBSD"

From RCS Wiki
Jump to navigation Jump to search
Line 3: Line 3:
 
'''Note:''' This documentation is intended to provide information to public users and be freely available.  However, knowledge was derived from the QEMU documentation, which is licensed GPL v2.  This is therefore distributed derivative works, and as such, this documentation is required to be licensed GPL v2.  Please be aware this supercedes the Creative Commons deed under which this wiki is published.
 
'''Note:''' This documentation is intended to provide information to public users and be freely available.  However, knowledge was derived from the QEMU documentation, which is licensed GPL v2.  This is therefore distributed derivative works, and as such, this documentation is required to be licensed GPL v2.  Please be aware this supercedes the Creative Commons deed under which this wiki is published.
  
−
=== The short version ===
+
=== QEMU: The short version ===
 
The goal is to rely on the default configuration options, thereby requiring as few custom options as possible.  Further configuration options are possible but beyond the scope of this tutorial.
 
The goal is to rely on the default configuration options, thereby requiring as few custom options as possible.  Further configuration options are possible but beyond the scope of this tutorial.
  
Line 29: Line 29:
 
</pre>
 
</pre>
  
−
Choose MBR for disk partition type.  Use DHCP to configure IPv4, and do not configure IPv6.  Choose any additional components as needed during the installation process, including the port and source code trees as desired.  For this example we named the machine "BEFreeBSD."  Add a user and add the user to group wheel (for this we created "user1").
+
The [[#FreeBSD installation details]] pick up the process at this point.  To remove the CDROM from the device tree, use
−
 
 
−
Either remove remote login by root at this time or do so after rebooting.  Shutdown the VM and relaunch it with
 
−
 
 
−
<pre>
 
−
# qemu-system-ppc64 --enable-kvm -M pseries,cap-nested-hv=true,cap-ibs=workaround -cpu host -smp cores=4,threads=4,sockets=1 -m 8G -nographic -nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:32,hostfwd=tcp::2222-:22 fbsd.raw
 
−
</pre>
 
−
 
 
−
to remove the CD installer.  Log in with the user ("user1") that was added to the wheel group.  Check the configuration for any unexpected values.
 
−
 
 
−
<pre>
 
−
% dmesg | grep CPU
 
−
FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs
 
−
xicp0: Handling CPUs 0-15
 
−
cpulist0: <Open Firmware CPU Group> on ofwbus0
 
−
cpu0: <Open Firmware CPU> on cpulist0
 
−
 
 
−
% dmesg | grep memory
 
−
real memory  = 8544366592 (8148 MB)
 
−
avail memory = 8216293376 (7835 MB)
 
−
 
 
−
% uname -a
 
−
FreeBSD BEFreeBSD 14.3-RELEASE FreeBSD 14.3-RELEASE releng/14.3-n271432-8c9ce319fef7 GENERIC powerpc
 
−
 
 
−
% ifconfig -a
 
−
em0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
 
−
        options=48525bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
 
−
        ether 52:54:98:76:54:32
 
−
        inet 10.0.2.15 netmask 0xffffff00 broadcast 10.0.2.255
 
−
        media: Ethernet autoselect (1000baseT <full-duplex>)
 
−
        status: active
 
−
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
 
−
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
 
−
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
 
−
        inet 127.0.0.1 netmask 0xff000000
 
−
        inet6 ::1 prefixlen 128
 
−
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
 
−
        groups: lo
 
−
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
 
−
</pre>
 
−
 
 
−
Log in from the Debian host with the created user ("user1") to check SSH:
 
  
 
<pre>
 
<pre>
−
# ssh -p 2222 user1@127.0.0.1
 
−
(user1@127.0.0.1) Password for user1@BEFreeBSD:
 
−
Last login: Sun Jun 29 09:02:48 2025 from xx
 
−
FreeBSD 14.3-RELEASE (GENERIC) releng/14.3-n271432-8c9ce319fef7
 
−
 
−
Welcome to FreeBSD!
 
−
...
 
−
</pre>
 
−
 
−
Check that su works:
 
−
 
 
<pre>
 
<pre>
−
% su root
+
# qemu-system-ppc64 --enable-kvm -nographic -M pseries,cap-nested-hv=true,cap-ibs=workaround -cpu host -smp cores=4,threads=4,sockets=1 -m 8G -nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:32,hostfwd=tcp::2222-:22 fbsd.raw
−
Password:
 
−
#
 
 
</pre>
 
</pre>
  
−
Log in remotely to make sure external ssh access is available.  Configure automatic restart of VM with reboot of host machine.
 
  
−
=== The long version ===
+
=== QEMU: The long version ===
  
 
This tutorial draws from the following web sites:
 
This tutorial draws from the following web sites:
Line 171: Line 116:
 
==== Device tree at baseline ====
 
==== Device tree at baseline ====
  
−
At this point, launch qemu-system-ppc for kicks:
+
At this point, launch qemu-system-ppc64 for kicks:
  
 
<pre>
 
<pre>
Line 259: Line 204:
 
7e646948 :  |  +-- epow-events
 
7e646948 :  |  +-- epow-events
 
7e646ae8 :  |-- cpus
 
7e646ae8 :  |-- cpus
−
7e646d70 :  |  +-- PowerPC,POWER9@0
+
7e646d70 :  |  +-- PowerPC,POWER9@0   <--- CPU
 
7e647830 :  |-- pci@800000020000000
 
7e647830 :  |-- pci@800000020000000
 
7e649e10 :  |  |-- vga@0
 
7e649e10 :  |  |-- vga@0
Line 270: Line 215:
 
7e64b110 :  |  |-- l-lan@71000002
 
7e64b110 :  |  |-- l-lan@71000002
 
7e64b4e0 :  |  +-- v-scsi@71000003
 
7e64b4e0 :  |  +-- v-scsi@71000003
−
7e671ca0 :  |      +-- disk
+
7e671ca0 :  |      +-- disk             <--- default storage
 
7e64b9a0 :  |-- memory@0
 
7e64b9a0 :  |-- memory@0
 
7e64bb98 :  |-- interrupt-controller
 
7e64bb98 :  |-- interrupt-controller
Line 323: Line 268:
 
7e646948 :  |  +-- epow-events
 
7e646948 :  |  +-- epow-events
 
7e646ae8 :  |-- cpus
 
7e646ae8 :  |-- cpus
−
7e646db0 :  |  |-- PowerPC,POWER9@0
+
7e646db0 :  |  |-- PowerPC,POWER9@0     <--- 4 thread CPU
−
7e6477e8 :  |  |-- PowerPC,POWER9@4
+
7e6477e8 :  |  |-- PowerPC,POWER9@4     <--- 4 thread CPU
−
7e648220 :  |  |-- PowerPC,POWER9@8
+
7e648220 :  |  |-- PowerPC,POWER9@8     <--- 4 thread CPU
−
7e648c58 :  |  +-- PowerPC,POWER9@c
+
7e648c58 :  |  +-- PowerPC,POWER9@c     <--- 4 thread CPU
 
7e649738 :  |-- pci@800000020000000
 
7e649738 :  |-- pci@800000020000000
 
7e64bd18 :  |  |-- vga@0
 
7e64bd18 :  |  |-- vga@0
Line 360: Line 305:
 
<div id=storage></div>
 
<div id=storage></div>
 
====Storage====
 
====Storage====
 +
Storage devices are added with the -drive shortcut.  This combines block device and related parameters.  These devices are attached at a different point than the default provided disc location.  The simplest option is a "file" with a specified file on the host as the media.  In this case, we are using the FreeBSD ISO installer.  The "if" (interface) uses a default "virtio" already provided by QEMU.  Other interfaces are possible if you create them manually.  "id" is used to identify this -drive uniquely.  Note that this id is _not_ passed to FreeBSD or the device tree, so it will not show up as "/dev/drive1" in FreeBSD.
  
 +
<pre>
 +
-drive  file=FreeBSD-14.3-RELEASE-powerpc-powerpc64-disc1.iso,media=cdrom,if=virtio,id=drive1
 +
</pre>
  
 
With CDROM:
 
With CDROM:
Line 409: Line 358:
  
 
==== NIC ====
 
==== NIC ====
 +
The netowrk card requires a few parameters that can have multiple options.  Use -nic to combine several -netdev features into one shortcut.  The network card is for a "user" as opposed to a "bridge" or "top."  e1000 is a well-known basis for ethernet cards in virtual machines and will be used here as well.  We have to specify the MAC address and the only pitfall here is choosing one that conflicts with a local machine.  Therefore, if you run multiple VMs locally, select appropriately unique MAC addresses.
 +
 +
The default is ipv4 and ipv6 are on.  If either are specified, the one omitted will be off.  Since the VM has no real interface to the network, all traffic has to be forwarded to the guest using the "hostfwd" parameter.  For this example, we forward external port 2222 to internal port 22 (ssh).  Other ports can be forwarded as well.  Please refer to the QEMU documentation for specifying a wider range of ports and other syntax.
 +
 +
<pre>
 +
-nic user,model=e1000,mac=52:54:98:76:54:32,ipv6=off,hostfwd=tcp::2222-:22
 +
</pre>
 +
 
With the network card:
 
With the network card:
  
Line 456: Line 413:
 
</pre>
 
</pre>
  
−
=== Installation details ===
+
==== Complete QEMU command ====
 +
With the device tree sufficiently described, the last addition to the qemu-system-ppc64 command line is the destination file for the installation.  Previously we created "fbsd.raw" with [[#qemu-img]] and we now add it _after_ all of the device tree specifications:
 +
 
 +
<pre>
 +
qemu-system-ppc64 --enable-kvm -nographic -M pseries,cap-nested-hv=true,cap-ibs=workaround -cpu host -smp cores=4,threads=4,sockets=1 -m 8G -drive  file=FreeBSD-14.3-RELEASE-powerpc-powerpc64-disc1.iso,media=cdrom,if=virtio,id=drive1 -nic user,model=e1000,mac=52:54:98:76:54:32,ipv6=off,hostfwd=tcp::2222-:22 fbsd.raw
 +
</pre>
 +
 
 +
If all has gone well, you should see
 +
 
 
[[File:FreeBSD_Installer.png|center|frame|FreeBSD Installer]]
 
[[File:FreeBSD_Installer.png|center|frame|FreeBSD Installer]]
  
 +
=== FreeBSD installation details ===
 +
Once the VM is fully configured the next step is installing FreeBSD. 
 +
 +
Most of the details of installing FreeBSD are beyond the scope of this tutorial and can be found in many locations.  When navigating the installer, use the arrow keys to move between options and the space bar to select that option.
 +
 +
Choose MBR for disk partition type.  This is important because the Open Firmware in QEMU will not read GUID-style formatting.
 +
 +
Use DHCP to configure IPv4, and do not configure IPv6 unless you have to.  If so, you will need to reconfigure the -nic card.  The easiest option is to not specify ipv4=on, in which case both IPv4 and IPv6 will be on.
 +
 +
Choose any additional components as needed during the installation process, including the port and source code trees as desired.  For this example we named the machine "BEFreeBSD."  Add a user and add the user to group wheel (for this we created "user1").
 +
 +
Either remove remote login by root at this time or do so after rebooting.  Shutdown the VM and relaunch it with
  
 
<pre>
 
<pre>
 +
# qemu-system-ppc64 --enable-kvm -nographic -M pseries,cap-nested-hv=true,cap-ibs=workaround -cpu host -smp cores=4,threads=4,sockets=1 -m 8G -nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:32,hostfwd=tcp::2222-:22 fbsd.raw
 +
</pre>
 +
 +
to remove the CD installer.  Log in with the user ("user1") that was added to the wheel group.  Check the configuration for any unexpected values.
 +
 +
<pre>
 +
% uname -a
 +
FreeBSD BEFreeBSD 14.3-RELEASE FreeBSD 14.3-RELEASE releng/14.3-n271432-8c9ce319fef7 GENERIC powerpc
 +
 +
% dmesg | grep CPU
 +
FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs
 +
xicp0: Handling CPUs 0-15
 +
cpulist0: <Open Firmware CPU Group> on ofwbus0
 +
cpu0: <Open Firmware CPU> on cpulist0
 +
 +
% dmesg | grep memory
 +
real memory  = 8544366592 (8148 MB)
 +
avail memory = 8216293376 (7835 MB)
 +
 
% uname -a
 
% uname -a
 
FreeBSD BEFreeBSD 14.3-RELEASE FreeBSD 14.3-RELEASE releng/14.3-n271432-8c9ce319fef7 GENERIC powerpc
 
FreeBSD BEFreeBSD 14.3-RELEASE FreeBSD 14.3-RELEASE releng/14.3-n271432-8c9ce319fef7 GENERIC powerpc
 +
 +
% ifconfig -a
 +
em0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
 +
        options=48525bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
 +
        ether 52:54:98:76:54:32
 +
        inet 10.0.2.15 netmask 0xffffff00 broadcast 10.0.2.255
 +
        media: Ethernet autoselect (1000baseT <full-duplex>)
 +
        status: active
 +
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
 +
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
 +
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
 +
        inet 127.0.0.1 netmask 0xff000000
 +
        inet6 ::1 prefixlen 128
 +
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
 +
        groups: lo
 +
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
 
</pre>
 
</pre>
 +
 +
Log in from the Debian host with the created user ("user1") to check SSH:
 +
 +
<pre>
 +
# ssh -p 2222 user1@127.0.0.1
 +
(user1@127.0.0.1) Password for user1@BEFreeBSD:
 +
Last login: Sun Jun 29 09:02:48 2025 from xx
 +
FreeBSD 14.3-RELEASE (GENERIC) releng/14.3-n271432-8c9ce319fef7
 +
 +
Welcome to FreeBSD!
 +
...
 +
</pre>
 +
 +
Check that su works:
 +
 +
<pre>
 +
% su root
 +
Password:
 +
#
 +
</pre>
 +
 +
Log in remotely to make sure external ssh access is available.  Configure automatic restart of VM with reboot of host machine.  There are descriptions of this process at
 +
 +
https://bbs.archlinux.org/viewtopic.php?id=295380
 +
https://bbs.archlinux.org/viewtopic.php?id=275467
 +
 +
Other discussions and manuals exist.
 +
 +
With the proper installation of big-endian FreeBSD running as a guest in a virtual machine provided by the little-endian Debian host, this concludes the tutorial. 
 +
  
  

Revision as of 03:44, 2 July 2025

This tutorial takes the reader from the bare minimum to launch a virtual machine with QEMU to a full installation of big-endian FreeBSD. The Raptor POWER9 host machine is running little-endian Debian. The short version is for experienced users just needing specifics, while the long version examines each parameter and shows the effect on the virtual machine and installation. Specific details in the short version are linked to explanations in the long version.

Note: This documentation is intended to provide information to public users and be freely available. However, knowledge was derived from the QEMU documentation, which is licensed GPL v2. This is therefore distributed derivative works, and as such, this documentation is required to be licensed GPL v2. Please be aware this supercedes the Creative Commons deed under which this wiki is published.

QEMU: The short version

The goal is to rely on the default configuration options, thereby requiring as few custom options as possible. Further configuration options are possible but beyond the scope of this tutorial.

Make sure #qemu-system-ppc64 and #qemu-img are installed. Use apt install if they are not.

Create a raw disk file to be the installation destination (in this case, a 120GB file named "fbsd.raw"): 

# qemu-img create fbsd.raw 120G
Formatting 'fbsd.raw', fmt=raw size=128849018880
# ls -sl
total 924700# ls -sl
2199132 -rw-r--r-- 1 root root 128849018880 Jun 30 13:19 fbsd.raw
 924696 -rw-r--r-- 1 root root    952018944 Jun 25 05:04 FreeBSD-14.3-RELEASE-powerpc-powerpc64-disc1.iso

Download the most recent version of big-endian FreeBSD from

https://download.freebsd.org/releases/powerpc/powerpc64/ISO-IMAGES/

Using the example of FreeBSD 14.3 as the iso-based installer, fbsd.raw as the destination #storage device, four #CPU cores with four threads per core, 8GB of #RAM, e1000 for the #NIC, no IPv6, and 2222 as the ssh port into the guest: 

# qemu-system-ppc64 --enable-kvm -M pseries,cap-nested-hv=true,cap-ibs=workaround -cpu host -smp cores=4,threads=4,sockets=1 -m 8G -nographic -drive  file=FreeBSD-14.3-RELEASE-powerpc-powerpc64-disc1.iso,media=cdrom,if=virtio,id=drive1 -nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:32,hostfwd=tcp::2222-:22 fbsd.raw

The #FreeBSD installation details pick up the process at this point. To remove the CDROM from the device tree, use 

<pre>
# qemu-system-ppc64 --enable-kvm -nographic -M pseries,cap-nested-hv=true,cap-ibs=workaround -cpu host -smp cores=4,threads=4,sockets=1 -m 8G -nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:32,hostfwd=tcp::2222-:22 fbsd.raw


QEMU: The long version

This tutorial draws from the following web sites:

https://www.qemu.org/docs/master/system/introduction.html

https://www.qemu.org/docs/master/system/invocation.html

https://www.qemu.org/docs/master/tools/qemu-img.html

https://www.qemu.org/docs/master/system/ppc/powernv.html

qemu-img

Although the focus of this tutorial is on #qemu-systm-ppc4, the installation does require a "storage" device for the FreeBSD installation. Using qemu-img, create a raw disk file to be the installation destination (in this case, 120GB): 

# qemu-img create fbsd.raw 120G
Formatting 'fbsd.raw', fmt=raw size=128849018880
# ls -sl
total 924700
     4 -rw-r--r-- 1 root root 128849018880 Jun 28 13:31 fbsd.raw
924696 -rw-r--r-- 1 root root    952018944 Jun 25 05:04 FreeBSD-14.3-RELEASE-powerpc-powerpc64-disc1.iso

The file is dynamically sized; the -s flag shows how many blocks the file actually uses.


qemu-system-ppc64

qemu-system-ppc64 is the QEMU executable used to provide a virtual machine on Raptor's hosts. To determine if you are running on a physical or virtual host, execute 

# facter | grep virtual
is_virtual => false
virtual => physical

This is real host. A virtual one would look like 

# facter | grep virtual
is_virtual => true
virtual => kvm

Baseline

QEMU needs several parameters to describe the virtual machine. There are some baseline settings that should be included before describing the VM. 

--enable-kvm -nographic

--enable-kvm starts a "keyboard-video-mouse" style VM. It invokes several default "host" flags. The -nographic means output will go to the console the VM is initiated from. If this flag is omitted, output will eventually be sent to a pseudo-tty console. It can still be viewed by attaching "screen" to it, but boot messages will be lost.

-nodefaults

A flag to discuss but not include is 

-nodefaults

This flags wipes out all of the defaults for various devices and is useful for being very specific about the device tree in the VM. For this tutorial, we want to use as many defaults as possible. Do not include this flag unless you are prepared to add devices manually.

CPU

QEMU provides many frameworks for replicating existing machines. This includes the p-Series from IBM, which is the same as the Raptor systems. We are not emulating a different CPU, just changing endianness on a POWER9 host. As such, two flags to be included are 

-M pseries,cap-nested-hv=true,cap-ibs=workaround -cpu host

RAM

QEMU allows for part or all of the physical RAM to be used for the VM. However, since there is another operating system running on the bare metal machine, QEMU will run out of memory if all of the physical RAM is allocated to the VM. Use a smaller partition. For this tutorial, we are using 8GB of RAM. 

-m 8G

Device tree at baseline

At this point, launch qemu-system-ppc64 for kicks: 

# qemu-system-ppc64 --enable-kvm -nographic -M pseries,cap-nested-hv=true,cap-ibs=workaround -cpu host -m 8G


SLOF **********************************************************************
QEMU Starting
 Build Date = May  3 2025 09:06:15
 FW Version = release 20220719
 Press "s" to enter Open Firmware.

Populating /vdevice methods
Populating /vdevice/vty@71000000
Populating /vdevice/nvram@71000001
Populating /vdevice/l-lan@71000002
Populating /vdevice/v-scsi@71000003
       SCSI: Looking for devices
          8200000000000000 CD-ROM   : "QEMU     QEMU CD-ROM      2.5+"
Populating /pci@800000020000000
                     00 0000 (D) : 1234 1111    qemu vga
                     00 0800 (D) : 1033 0194    serial bus [ usb-xhci ]
No NVRAM common partition, re-initializing...
Installing QEMU fb



Scanning USB 
  XHCI: Initializing
    USB Keyboard 
    USB mouse 
Using default console: /vdevice/vty@71000000
     
  Welcome to Open Firmware

  Copyright (c) 2004, 2017 IBM Corporation All rights reserved.
  This program and the accompanying materials are made available
  under the terms of the BSD License available at
  http://www.opensource.org/licenses/bsd-license.php


Trying to load:  from: disk ... 
E3405: No such device
Trying to load:  from: /vdevice/v-scsi@71000003/disk@8200000000000000 ... No medium !

E3405: No such device
Trying to load:  from: /vdevice/l-lan@71000002 ... 
 Initializing NIC
  Reading MAC address from device: 52:54:00:12:34:56
  Requesting information via DHCP: done
  Using IPv4 address: 10.0.2.15
  Requesting file "" via TFTP from 10.0.2.2
Trying pxelinux.cfg files...
  TFTP error: TFTP access violation
  TFTP error: TFTP access violation
  TFTP error: TFTP access violation
  TFTP error: TFTP access violation
  TFTP error: TFTP access violation
  TFTP error: TFTP access violation
  TFTP error: TFTP access violation
  TFTP error: TFTP access violation
  TFTP error: TFTP access violation
  TFTP error: TFTP access violation

E3407: Load failed

  Type 'boot' and press return to continue booting the system.
  Type 'reset-all' and press return to reboot the system.


Ready! 
0 >

Without any boot media or a network card for TFTP, load will fail. However, this is a good time to examine the baseline device tree QEMU provides:

No devices added (default SLOF boot): 

0 > dev / ls  
7e644530 :  /
7e644b40 :  |-- ibm,persistent-memory
7e644da0 :  |-- hypervisor
7e644f00 :  |-- chosen
7e645310 :  |-- rtas
7e646638 :  |-- event-sources
7e6467f0 :  |   |-- hot-plug-events
7e646948 :  |   +-- epow-events
7e646ae8 :  |-- cpus
7e646d70 :  |   +-- PowerPC,POWER9@0    <--- CPU
7e647830 :  |-- pci@800000020000000
7e649e10 :  |   |-- vga@0
7e64a328 :  |   +-- usb-xhci@1
7e68d8f0 :  |       |-- usb-keyboard@1
7e68e3b0 :  |       +-- usb-mouse@2
7e64aa00 :  |-- vdevice
7e64ac80 :  |   |-- vty@71000000
7e64aea0 :  |   |-- nvram@71000001
7e64b110 :  |   |-- l-lan@71000002
7e64b4e0 :  |   +-- v-scsi@71000003
7e671ca0 :  |       +-- disk             <--- default storage 
7e64b9a0 :  |-- memory@0
7e64bb98 :  |-- interrupt-controller
7e64ecd8 :  |-- aliases
7e64ede0 :  |-- options
7e64eea0 :  |-- openprom
7e64efa8 :  |-- packages
7e64f070 :  |   |-- deblocker
7e64fb10 :  |   |-- disk-label
7e6541b0 :  |   |-- fat-files
7e656928 :  |   |-- rom-files
7e657348 :  |   |-- ext2-files
7e659dc0 :  |   |-- obp-tftp
7e65a190 :  |   +-- iso-9660
7e65ebd8 :  +-- mmu ok
0 >


SMP

Notice that the device tree only includes a single PowerPC,POWER9 instance. Add 16-core SMP capacity to the cpu with 

-smp cores=4,threads=4,sockets=1

Now the device tree looks like 

# qemu-system-ppc64 --enable-kvm -nographic -M pseries,cap-nested-hv=true,cap-ibs=workaround -cpu host -smp cores=4,threads=4,sockets=1 -m 8G 


SLOF **********************************************************************
QEMU Starting

....
  TFTP error: TFTP access violation

E3407: Load failed

  Type 'boot' and press return to continue booting the system.
  Type 'reset-all' and press return to reboot the system.
0 > dev / ls  
7e644530 :  /
7e644b40 :  |-- ibm,persistent-memory
7e644da0 :  |-- hypervisor
7e644f00 :  |-- chosen
7e645310 :  |-- rtas
7e646638 :  |-- event-sources
7e6467f0 :  |   |-- hot-plug-events
7e646948 :  |   +-- epow-events
7e646ae8 :  |-- cpus
7e646db0 :  |   |-- PowerPC,POWER9@0     <--- 4 thread CPU
7e6477e8 :  |   |-- PowerPC,POWER9@4     <--- 4 thread CPU
7e648220 :  |   |-- PowerPC,POWER9@8     <--- 4 thread CPU
7e648c58 :  |   +-- PowerPC,POWER9@c     <--- 4 thread CPU
7e649738 :  |-- pci@800000020000000
7e64bd18 :  |   |-- vga@0
7e64c230 :  |   +-- usb-xhci@1
7e68f7f8 :  |       |-- usb-keyboard@1
7e6902b8 :  |       +-- usb-mouse@2
7e64c908 :  |-- vdevice
7e64cb88 :  |   |-- vty@71000000
7e64cda8 :  |   |-- nvram@71000001
7e64d018 :  |   |-- l-lan@71000002
7e64d3e8 :  |   +-- v-scsi@71000003
7e673ba8 :  |       +-- disk
7e64d8a8 :  |-- memory@0
7e64daa0 :  |-- interrupt-controller
7e650be0 :  |-- aliases
7e650ce8 :  |-- options
7e650da8 :  |-- openprom
7e650eb0 :  |-- packages
7e650f78 :  |   |-- deblocker
7e651a18 :  |   |-- disk-label
7e6560b8 :  |   |-- fat-files
7e658830 :  |   |-- rom-files
7e659250 :  |   |-- ext2-files
7e65bcc8 :  |   |-- obp-tftp
7e65c098 :  |   +-- iso-9660
7e660ae0 :  +-- mmu ok
0 >

Note the increments of 4 in the instances.


Storage

Storage devices are added with the -drive shortcut. This combines block device and related parameters. These devices are attached at a different point than the default provided disc location. The simplest option is a "file" with a specified file on the host as the media. In this case, we are using the FreeBSD ISO installer. The "if" (interface) uses a default "virtio" already provided by QEMU. Other interfaces are possible if you create them manually. "id" is used to identify this -drive uniquely. Note that this id is _not_ passed to FreeBSD or the device tree, so it will not show up as "/dev/drive1" in FreeBSD. 

-drive  file=FreeBSD-14.3-RELEASE-powerpc-powerpc64-disc1.iso,media=cdrom,if=virtio,id=drive1

With CDROM: 

Ready! 
0 > dev / ls  
7e644530 :  /
7e644b40 :  |-- ibm,persistent-memory
7e644da0 :  |-- hypervisor
7e644f00 :  |-- chosen
7e645310 :  |-- rtas
7e646638 :  |-- event-sources
7e6467f0 :  |   |-- hot-plug-events
7e646948 :  |   +-- epow-events
7e646ae8 :  |-- cpus
7e646db0 :  |   |-- PowerPC,POWER9@0
7e6477e8 :  |   |-- PowerPC,POWER9@4
7e648220 :  |   |-- PowerPC,POWER9@8
7e648c58 :  |   +-- PowerPC,POWER9@c
7e649738 :  |-- pci@800000020000000
7e64bd18 :  |   |-- ethernet@0
7e64c280 :  |   |-- vga@1
7e64c798 :  |   |-- usb-xhci@2
7e691d68 :  |   |   |-- usb-keyboard@1
7e692828 :  |   |   +-- usb-mouse@2
7e64cdc8 :  |   +-- scsi@3           <--- added device
7e64d428 :  |-- vdevice
7e64d6a8 :  |   |-- vty@71000000
7e64d8c8 :  |   |-- nvram@71000001
7e64db38 :  |   +-- v-scsi@71000002
7e673d00 :  |       +-- disk
7e64dff8 :  |-- memory@0
7e64e1f0 :  |-- interrupt-controller
7e651330 :  |-- aliases
7e651438 :  |-- options
7e6514f8 :  |-- openprom
7e651600 :  |-- packages
7e6516c8 :  |   |-- deblocker
7e652168 :  |   |-- disk-label
7e656808 :  |   |-- fat-files
7e658f80 :  |   |-- rom-files
7e6599a0 :  |   |-- ext2-files
7e65c418 :  |   |-- obp-tftp
7e65c7e8 :  |   +-- iso-9660
7e661230 :  +-- mmu ok

NIC

The netowrk card requires a few parameters that can have multiple options. Use -nic to combine several -netdev features into one shortcut. The network card is for a "user" as opposed to a "bridge" or "top." e1000 is a well-known basis for ethernet cards in virtual machines and will be used here as well. We have to specify the MAC address and the only pitfall here is choosing one that conflicts with a local machine. Therefore, if you run multiple VMs locally, select appropriately unique MAC addresses.

The default is ipv4 and ipv6 are on. If either are specified, the one omitted will be off. Since the VM has no real interface to the network, all traffic has to be forwarded to the guest using the "hostfwd" parameter. For this example, we forward external port 2222 to internal port 22 (ssh). Other ports can be forwarded as well. Please refer to the QEMU documentation for specifying a wider range of ports and other syntax. 

-nic user,model=e1000,mac=52:54:98:76:54:32,ipv6=off,hostfwd=tcp::2222-:22

With the network card: 

0 > dev /    ok
0 > ls  
7e644530 :  /
7e644b40 :  |-- ibm,persistent-memory
7e644da0 :  |-- hypervisor
7e644f00 :  |-- chosen
7e645310 :  |-- rtas
7e646638 :  |-- event-sources
7e6467f0 :  |   |-- hot-plug-events
7e646948 :  |   +-- epow-events
7e646ae8 :  |-- cpus
7e646db0 :  |   |-- PowerPC,POWER9@0
7e6477e8 :  |   |-- PowerPC,POWER9@4
7e648220 :  |   |-- PowerPC,POWER9@8
7e648c58 :  |   +-- PowerPC,POWER9@c
7e649738 :  |-- pci@800000020000000
7e64bd18 :  |   |-- ethernet@0          <-- Added device
7e64c280 :  |   |-- vga@1
7e64c798 :  |   |-- usb-xhci@2
7e6933f8 :  |   |   |-- usb-keyboard@1
7e693eb8 :  |   |   +-- usb-mouse@2
7e64cdc8 :  |   |-- scsi@3
7e64d380 :  |   +-- scsi@4
7e64d9e0 :  |-- vdevice
7e64dc60 :  |   |-- vty@71000000
7e64de80 :  |   |-- nvram@71000001
7e64e0f0 :  |   +-- v-scsi@71000002
7e6742b8 :  |       +-- disk
7e64e5b0 :  |-- memory@0
7e64e7a8 :  |-- interrupt-controller
7e6518e8 :  |-- aliases
7e6519f0 :  |-- options
7e651ab0 :  |-- openprom
7e651bb8 :  |-- packages
7e651c80 :  |   |-- deblocker
7e652720 :  |   |-- disk-label
7e656dc0 :  |   |-- fat-files
7e659538 :  |   |-- rom-files
7e659f58 :  |   |-- ext2-files
7e65c9d0 :  |   |-- obp-tftp
7e65cda0 :  |   +-- iso-9660
7e6617e8 :  +-- mmu ok

Complete QEMU command

With the device tree sufficiently described, the last addition to the qemu-system-ppc64 command line is the destination file for the installation. Previously we created "fbsd.raw" with #qemu-img and we now add it _after_ all of the device tree specifications: 

qemu-system-ppc64 --enable-kvm -nographic -M pseries,cap-nested-hv=true,cap-ibs=workaround -cpu host -smp cores=4,threads=4,sockets=1 -m 8G -drive  file=FreeBSD-14.3-RELEASE-powerpc-powerpc64-disc1.iso,media=cdrom,if=virtio,id=drive1 -nic user,model=e1000,mac=52:54:98:76:54:32,ipv6=off,hostfwd=tcp::2222-:22 fbsd.raw

If all has gone well, you should see

FreeBSD Installer

FreeBSD installation details

Once the VM is fully configured the next step is installing FreeBSD.

Most of the details of installing FreeBSD are beyond the scope of this tutorial and can be found in many locations. When navigating the installer, use the arrow keys to move between options and the space bar to select that option.

Choose MBR for disk partition type. This is important because the Open Firmware in QEMU will not read GUID-style formatting.

Use DHCP to configure IPv4, and do not configure IPv6 unless you have to. If so, you will need to reconfigure the -nic card. The easiest option is to not specify ipv4=on, in which case both IPv4 and IPv6 will be on.

Choose any additional components as needed during the installation process, including the port and source code trees as desired. For this example we named the machine "BEFreeBSD." Add a user and add the user to group wheel (for this we created "user1").

Either remove remote login by root at this time or do so after rebooting. Shutdown the VM and relaunch it with 

# qemu-system-ppc64 --enable-kvm -nographic -M pseries,cap-nested-hv=true,cap-ibs=workaround -cpu host -smp cores=4,threads=4,sockets=1 -m 8G -nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:32,hostfwd=tcp::2222-:22 fbsd.raw

to remove the CD installer. Log in with the user ("user1") that was added to the wheel group. Check the configuration for any unexpected values. 

% uname -a
FreeBSD BEFreeBSD 14.3-RELEASE FreeBSD 14.3-RELEASE releng/14.3-n271432-8c9ce319fef7 GENERIC powerpc

% dmesg | grep CPU
FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs
xicp0: Handling CPUs 0-15
cpulist0: <Open Firmware CPU Group> on ofwbus0
cpu0: <Open Firmware CPU> on cpulist0

 % dmesg | grep memory
real memory  = 8544366592 (8148 MB)
avail memory = 8216293376 (7835 MB)

% uname -a
FreeBSD BEFreeBSD 14.3-RELEASE FreeBSD 14.3-RELEASE releng/14.3-n271432-8c9ce319fef7 GENERIC powerpc

% ifconfig -a
em0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=48525bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,HWSTATS,MEXTPG>
        ether 52:54:98:76:54:32
        inet 10.0.2.15 netmask 0xffffff00 broadcast 10.0.2.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Log in from the Debian host with the created user ("user1") to check SSH: 

# ssh -p 2222 user1@127.0.0.1
(user1@127.0.0.1) Password for user1@BEFreeBSD:
Last login: Sun Jun 29 09:02:48 2025 from xx
FreeBSD 14.3-RELEASE (GENERIC) releng/14.3-n271432-8c9ce319fef7

Welcome to FreeBSD!
...

Check that su works: 

% su root
Password:
#

Log in remotely to make sure external ssh access is available. Configure automatic restart of VM with reboot of host machine. There are descriptions of this process at

https://bbs.archlinux.org/viewtopic.php?id=295380 https://bbs.archlinux.org/viewtopic.php?id=275467

Other discussions and manuals exist.

With the proper installation of big-endian FreeBSD running as a guest in a virtual machine provided by the little-endian Debian host, this concludes the tutorial.


Post-boot blues

Mountroot fails

mountroot: waiting for device /dev/vtbd1s2a...
Mounting from ufs:/dev/vtbd1s2a failed with error 19.

Loader variables:
  vfs.root.mountfrom=ufs:/dev/vtbd1s2a
  vfs.root.mountfrom.options=rw

Solution: There is a mismatch in the device tree configuration and the boot configuration. This can occur when the installation is done with a -device but the reboot is done with a file. FreeBSD and QEMU don't always agree on the order of the devices, and FreeBSD is looking for the hard drive where it was, not where it is. The following will get FreeBSD up and running from a file: 

mountroot> ?

List of GEOM managed disk devices:
  ufsid/6860fe636e2c3598 da0s2b da0s2a da0s2 da0s1 da0 cd0

mountroot> ufs:/dev/da0s2

to specify that da0s2 is a ufs file system, and then after booting edit /etc/fstab: 

# cat /etc/fstab
# Device        Mountpoint      FStype  Options Dump    Pass#
/dev/da0s2a   /               ufs     rw      1       1
/dev/da0s2b   none            swap    sw      0       0

Some other errors

QEMU's output is somewhere else.

At the end of the output from QEMU, further output is suppressed and the last line is 

char device redirected to /dev/pts/N (label monitor-base)

Solution: Output has now been redirected to /dev/pts/N where N is a number correlated to the pseudo-tty in /dev/pts/. You can attach to it with 

# screen /dev/pts/N 115200 8 N 1

or add -nographic to the command line.


Duplicate/conflicting serial devices

qemu-system-ppc64: Duplicate ID 'serial0' for chardev
qemu-system-ppc64: could not connect serial device to character backend 'vc:80Cx24C'

Solution: Most likely you are attempting to add a chardev with the id=serial0 parameter. vc is the default i/o and is named "serial0." Use -nodefaults and provide your own chardev (on the command line) or use the default serial0.


No host

 qemu-system-ppc64: unable to find CPU model 'host'

Solution: The command line is missing --enable-kvm.


No discs

Trying to load:  from: disk ... 
E3405: No such device
Trying to load:  from: /vdevice/v-scsi@71000003/disk@8200000000000000 ... No medium !

Solution: No virtio has been specified in the command line. Add one, or use the default vblk, which can be invoked by separating the raw file from the parameter list.


Can not use IDE on pSeries VMs

qemu-system-ppc64: -drive file=FreeBSD-14.3-RELEASE-powerpc-powerpc64-disc1.iso,if=ide,index=1,media=cdrom: machine type does not support if=ide,bus=0,unit=1
qemu-system-ppc64: -drive file=fbsd.raw,if=ide,index=0,format=raw: machine type does not support if=ide,bus=0,unit=0

Solution: Do not use ide with a pseries virtual machine. It is easier to use the defaults unless you have specific needs. SCSI should work.

Retrieved from "https://wiki.raptorcs.com/w/index.php?title=From_QEMU_to_BigEndian_FreeBSD&oldid=4892"