Debricking the BMC

While these instructions have been successfully applied in practice, they are still preliminary. Ask questions in IRC if you are unclear on what to do!

In the event of a failure updating the BMC, but with a functioning u-boot, you can still recover by using U-Boot to manually bootstrap the BMC by manually loading a boot image over the network or BMC serial line.

If your BMC flash is corrupted to the extent that U-Boot is not loading properly, you WILL need to remove and flash the BMC flash chip externally.

• Prepare a TFTP server, and place image-bmc, image-rofs, and image-kernel in the root. (TODO: elaborate on how to set this up)

• Connect a serial console to the BMC serial port (J7701, serial port bracket required) and set to 115200 8n1, disable RTS/CTS (hardware flow control).
• Disconnect and reconnect power to the machine to force a BMC restart. Press a key to interrupt auto-boot when prompted.
• Run dhcp x.x.x.x:image-bmc, replacing the IP address of your TFTP server. This will load a copy of the stock boot image into RAM.
• Run bootm 83080000. This will prepare and boot off of the loaded virtual image.
• If your rofs partition is not functional, you will be dropped into the systemd emergency shell at this point. Try both the password you set as well as the default 0penBmc, it may be one or the other depending on the state of the rwfs partition. If it boots up properly instead of dropping you into the emergency shell, the problem is probably in your kernel partition and you can retry flashing your image-kernel using the normal procedure. (The rest of these instructions are for the systemd emergency shell.)
• mount -t tmpfs none /tmp
• run udhcpc to get an IP address. (TODO: verify that this is the actual command that you run. Do you have to specify the network interface too?)
• cd /tmp
• tftp -g -r image-rofs x.x.x.x
• tftp -g -r image-kernel x.x.x.x
• IMPORTANT: Use md5sum, sha1sum, or sha256sum to verify successful transfer of image-rofs and image-kernel! tftp is a very barebones protocol and relies on transport layer checksumming, which is optional and not always available in UDP!
• Verify that the output of cat /sys/class/mtd/mtd3/name is kernel and the output of cat /sys/class/mtd/mtd4/name is rofs. We will be flashing mtd partitions directly in the next step and this is the last chance to verify that they will be flashed to the correct partition.
• flashcp -v image-kernel /dev/mtd3
• flashcp -v image-rofs /dev/mtd4
• (TODO: Describe how to reset rwfs in case it was damaged as well?) note: the kernel param for bypassing rwfs is "overlay-filesystem-in-ram". Append it to the existing boot-args before running the bootm command. This can also be used as part of a password reset procedure.
• After the flash is complete, you can run restart the BMC and it should boot successfully.

• (TODO: Discussion of using Kermit to upload the image without network access) note: I (Bdragon) have successfully done a ram-only boot using cu's built in xmodem support (escape sequence ~X) to do an image transfer into RAM over the BMC serial interface.
• (TODO: Discuss using u-boot's built in cmp tool to perform basic validation of the u-boot image against a second copy loaded into RAM.)
• (TODO: Write a u-boot standalone application to disable the AST watchdog, and write instructions for loading and executing it from the u-boot shell (the "go" command), to work around the cold-boot watchdog issue.)
• (TODO: Load recovery images over USB?) note: The onboard USB port is connected to the USB switch after all, so this might be problematic.
• (TODO: Discussion of u-boot memory map) Short version is: flash lives at 0x20000000 and default base address for the memory loading tools is 0x83000000. So add 0x63000000 to any flash address to get the eqivilent address for an image-bmc file loaded into RAM. For example, the bootable image of a loaded image-bmc is at 0x83080000.