Booting My Own Linux 7.3 Build on a BeagleBone Black

中文版：在 BeagleBone Black 上把自己編的 Linux 7.3 開起來

I have a BeagleBone Black (TI AM335x, Cortex-A8, 512 MiB) whose SD card runs the official Debian 13 image with a 6.12 kernel. The goal was simple: cross-compile mainline Linux 7.3-rc7 on my desktop, hand it to the board over TFTP, and have it boot to a login prompt with working networking.

That sounds like a handful of commands. In practice I hit five problems along the way, and only one of them had anything to do with the kernel. This post walks through them in the order they happened.

Setup

Role	Details
Host	x86 desktop (28 threads), Ubuntu, IP 192.168.10.154
Target	BeagleBone Black, U-Boot 2019.04, Debian 13 + 6.12.45-bone34 on SD
Console	J1 debug UART to a CP2102 (3.3 V), 115200 8N1, /dev/ttyUSB0
Network	Board eth0 and host on the same 192.168.10.0/24

The whole path looks like this:

Host                                    BeagleBone Black
────                                    ────────────────
make zImage + dtb                       ROM → SPL → U-Boot
   │                                       │  (interrupt autoboot)
   └─► TFTP root ──── tftp ──────────────► load to 0x82000000 / 0x88000000
                                           │
serial_console.py ◄──── UART ───────────── bootz → Linux 7.3
                                           │
                                           └─ mount the existing rootfs on SD

Step 1: Build the kernel

The host had no ARM Linux cross toolchain, only the bare-metal arm-none-eabi one, so:

sudo apt install gcc-arm-linux-gnueabihf     # GCC 13.3

I already had a mainline tree, but it carries an unrelated development branch I didn't want to disturb. So I created a detached worktree on the tag and built out of tree:

git -C ~/workspace/linux-dts worktree add --detach ~/workspace/bbb-bringup/linux v7.3-rc7
cd ~/workspace/bbb-bringup/linux

M="make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- O=../build-arm"
$M omap2plus_defconfig
scripts/config --file ../build-arm/.config --set-str LOCALVERSION "-bringup"
$M olddefconfig
$M -j28 zImage ti/omap/am335x-boneblack.dtb

• omap2plus_defconfig is the shared defconfig for the TI OMAP/AM335x family.
• LOCALVERSION=-bringup makes the version string 7.3.0-rc7-bringup, so after boot it's obvious whether my kernel is the one running.
• Newer kernels moved the TI device trees to arch/arm/boot/dts/ti/omap/, so the dtb target has to be ti/omap/am335x-boneblack.dtb.

With 28 threads the build took 1 min 51 s with zero warnings. The resulting zImage is 5.7 MB, noticeably smaller than Debian's 8.8 MB, since Debian's config enables far more modules and features.

Do I need an initrd?

The initrd and /lib/modules on the SD card are built for 6.12, so they can't be used with a 7.3 kernel anyway. I checked that everything needed to reach the rootfs is built in:

CONFIG_MMC_OMAP_HS=y          # SD/eMMC controller
CONFIG_EXT4_FS=y              # rootfs filesystem
CONFIG_SERIAL_8250_OMAP=y     # console
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_TI_CPSW_SWITCHDEV=y    # Ethernet
CONFIG_DEVTMPFS_MOUNT=y       # /dev without an initramfs

All =y, so the kernel can mount the rootfs on SD directly, no initramfs required.

Step 2: TFTP server

The host didn't have a TFTP server running. It did have dnsmasq, so I enabled only its TFTP part: --port=0 turns DNS off, DHCP stays off, and --interface plus --bind-interfaces limit it to the NIC facing the board.

dnsmasq -k --port=0 --enable-tftp --tftp-root=$HOME/workspace/bbb-bringup/tftp \
        --tftp-secure --interface=enp5s0 --bind-interfaces

The TFTP root holds two sets of files: the stock kernel copied back from the board as a control, and my own zImage-bringup and am335x-boneblack-bringup.dtb. When something fails, I can switch back to the stock set to tell a kernel problem from an environment problem.

Problem 1: bootdelay=0, so autoboot can't be interrupted

This board's U-Boot environment has bootdelay=0. It prints a single Press SPACE to abort autoboot in 0 seconds line and moves on; there's no way to hit a key by hand.

With bootdelay=0, U-Boot still checks the UART for input once, so the fix is to have a space already sitting in the UART receive FIFO before it checks. I wrote a small tool, serial_console.py, that timestamps every UART line into a log file and, while a flag file exists, sends a space every 20 ms, then stops and deletes the flag as soon as it sees U-Boot's => prompt.

The first version spammed spaces from power-on onward, which ended up in Linux's login prompt. I changed it to send only after seeing a U-Boot marker (U-Boot 20xx, Board:, DRAM:) and before Starting kernel, so Linux never sees any of it.

The tool had a bug of its own. To decide whether we were in U-Boot, it only searched the last 64 bytes for the banner, but the CP2102 delivers several hundred bytes per read, so the banner was pushed out before it was ever checked. Two power cycles in a row missed the prompt. The fix was to scan each incoming chunk in full, and before asking for another power cycle I simulated the scenarios on a pty acting as a fake serial port. Every power cycle needs a human to pull the plug, so it's cheaper to simulate in software first.

Problem 2: USB hub re-enumeration eats the start of the log

The CP2102 and the BBB were plugged into the same USB hub. When the BBB powers on, the whole hub drops off and re-enumerates, and /dev/ttyUSB0 disappears for 2 to 4 seconds:

cp210x ttyUSB0: cp210x converter now disconnected from ttyUSB0
usb 1-4: new high-speed USB device number 18 using xhci_hcd
...
usb 1-4.1: cp210x converter now attached to ttyUSB0

That caused two problems:

1. SPL finishes printing within those few seconds, so it is never captured.
2. Once the hub took 4 seconds to come back. By the time the logger reopened the port, U-Boot was already printing Board: BeagleBone Black, the banner was long gone, and not a single space was sent.

The software workaround: when the port reopens and the flag is still set, send spaces blindly for 4 seconds without waiting for any marker. The real fix is hardware: plug the UART adapter into a motherboard USB port instead of sharing a hub with the board, or power the BBB from its DC jack.

Problem 3: Packets arrive, replies leave through Tailscale

After interrupting U-Boot, both ping and tftp timed out:

=> ping 192.168.10.154
ping failed; host 192.168.10.154 is not alive
=> tftp ${loadaddr} zImage
Loading: *
T T T T

A packet capture on the host showed the board's ARP, ICMP echo request, and TFTP RRQ all arriving, yet the host never sent a single reply:

ARP, Request who-has 192.168.10.154 tell 192.168.10.242
ARP, Reply 192.168.10.154 is-at aa:bb:cc:dd:ee:ff
IP 192.168.10.242 > 192.168.10.154: ICMP echo request, id 0, seq 3, length 8
IP 192.168.10.242.3040 > 192.168.10.154.69: TFTP, length 38, RRQ "zImage" octet

The firewall's INPUT policy was accept, so nothing was being blocked. The return route told the story:

$ ip route get 192.168.10.242
192.168.10.242 dev tailscale0 table 52 src 100.x.y.z

Some machine on my tailnet advertises a 192.168.10.0/24 subnet route. Tailscale's policy rule (priority 5270, lookup 52) sits ahead of the main table (32766), so every reply to the board was being sent into tailscale0. A higher-priority rule sends the local LAN back through the main table:

sudo ip rule add to 192.168.10.0/24 lookup main priority 2500

After that, tftp ran at 5.8 MiB/s. When requests arrive but replies vanish, run ip route get to see which interface the reply actually takes; it's much faster than suspecting the firewall or the board. The rule doesn't survive a reboot, so the TFTP boot script now checks for it before starting.

Step 3: Boot manually from U-Boot

With autoboot interrupted and the network working, in U-Boot:

=> setenv ipaddr 192.168.10.242; setenv serverip 192.168.10.154
=> tftp ${loadaddr} zImage-bringup                 # 0x82000000, 5694896 bytes
=> tftp ${fdtaddr}  am335x-boneblack-bringup.dtb   # 0x88000000, 98486 bytes
=> md.l ${loadaddr} 0xc                            # offset 0x24 must be 016f2818
=> md.b ${fdtaddr} 4                               # must be d0 0d fe ed
=> setenv bootargs console=ttyS0,115200n8 root=/dev/mmcblk0p3 ro rootfstype=ext4 rootwait earlycon bringup.boot=tftp-own
=> bootz ${loadaddr} - ${fdtaddr}

• The middle argument of bootz is the initrd; - means none.
• bringup.boot=tftp-own is a marker of my own. The kernel doesn't recognize it and leaves it untouched in /proc/cmdline, which proves the boot used my bootargs.
• Checking the zImage magic and FDT magic with md first avoids wasting a boot on files that never loaded correctly.

Boot log:

Starting kernel ...
[    0.000000] Linux version 7.3.0-rc7-bringup (alanhc@alanhc-14700) (arm-linux-gnueabihf-gcc ... 13.3.0 ...)
[    0.000000] Kernel command line: console=ttyS0,115200n8 root=/dev/mmcblk0p3 ro ... bringup.boot=tftp-own
[    2.601037] mmcblk0: mmc0:59b4 USDU1 14.9 GiB
[    4.498186] VFS: Mounted root (ext4 filesystem) readonly on device 179:3.
[    4.530783] Run /sbin/init as init process
...
BeagleBone login:

It reached a login prompt. But the Web console banner line didn't show an IP address.

Problem 4: Login prompt, broken network

ping didn't work. The log had this line:

cpsw-switch 4a100000.switch end0: renamed from eth0

The stock bootargs include net.ifnames=0, and I had dropped it while trimming the bootargs by hand. Without it, systemd applies predictable network interface names and renames eth0 to end0. The rootfs network configuration is written for eth0, so no address was ever assigned.

Going back through earlier logs, the TFTP boots of the stock 6.12 kernel had the same end0: renamed from eth0 line. Networking was broken there too; I just hadn't pinged, because I was only checking whether it reached login.

With net.ifnames=0 restored:

cpsw-switch 4a100000.switch eth0: Link is Up - 100Mbps/Full - flow control off
BeagleBone login:

It answered ping within a second of reaching login.

Two lessons: find out what a bootargs parameter does before deleting it, and "reaches login" is not the same as "boots successfully". Decide the acceptance criteria up front.

Problem 5: Soft reboot hangs

To avoid asking someone to pull the power every time, I tried systemctl reboot over SSH. The stock 6.12 system hung at the same spot both times:

[  OK  ] Stopped ssh.service - OpenBSD Secure Shell server.
[  OK  ] Stopped serial-getty@ttyGS0.service - Serial Getty on ttyGS0.

Nothing else was printed, yet the board still answered ping. The first time I suspected my space spamming. The second time I was sure no spaces were sent during the Linux phase, and it hung on the same line, so the problem is in the system itself. I haven't chased it yet; it's the next exercise, starting with journalctl -b -1 from the previous boot or reproducing it with systemd.log_level=debug.

Acceptance

The final power cycle ran fully automatically, from interrupting U-Boot to the checks. Then over SSH:

$ uname -r
7.3.0-rc7-bringup
$ cat /proc/cmdline
console=ttyS0,115200n8 root=/dev/mmcblk0p3 ro rootfstype=ext4 rootwait earlycon net.ifnames=0 bringup.boot=tftp-own
$ ip -4 -br addr show eth0
eth0             UP             192.168.10.242/24
$ systemctl --failed
● bb-usb-gadgets.service loaded failed failed BeagleBoard.org USB gadgets
● iwd.service            loaded failed failed Wireless service

Only the USB gadget and Wi-Fi services failed, both because the rootfs has no 7.3 modules, which was expected.

Compared with the stock boot:

	Stock 6.12 + initrd	Own 7.3, no initrd
Kernel size	8.8 MB	5.7 MB
rootfs mounted	12.6 s	3.7 s
Starting kernel to login	~58 s	~42 s
Network	Works	Works (needs net.ifnames=0)
Failed units	None	bb-usb-gadgets, iwd (missing modules)

The rootfs mounts almost 9 seconds sooner, mostly from skipping the initramfs and the module loading inside it.

Looking back

Of the five problems, only net.ifnames=0 touched the kernel, and strictly speaking that's a userspace naming policy, not a kernel bug. The other four were:

• Bootloader configuration: bootdelay=0.
• USB topology: the UART adapter sharing a hub with the board.
• Host networking: a Tailscale subnet route hijacking the return path.
• Stock rootfs: soft reboot hangs.

I think that's what bring-up usually looks like. Getting the kernel to compile is only the starting point; most of the time goes into making the whole chain observable and reproducible. The investments that paid off this time were the timestamped serial logger, the habit of checking magic numbers before booting, and defining acceptance criteria for every step (uname -r, /proc/cmdline, ping).

Next steps

• Install the 7.3 modules into the rootfs (make modules_install INSTALL_MOD_PATH=...) to bring USB gadget and Wi-Fi back.
• Move the rootfs to NFS so the SD card is out of the loop entirely.
• Track down why the stock 6.12 system hangs on soft reboot.
• Disable a node in the device tree on purpose and see how the kernel log reacts.