The start hangs were memory pressure
On 32 GB of RAM, about one start in four hung. With graph capture and MTP on, 5 of 19 starts stalled while both cards copied the weights, and the kernel logged GPU job timeouts. Today I moved to 64 GB and it stopped.
What I saw before: every start used up nearly all the host RAM. MemAvailable dropped to 0.3 GB at the peak and each start pushed about 5 GB into swap. Each card holds roughly 11 GB of pinned host memory once the graphs are captured, and about 23 GB at the peak during load. Two cards is most of a 32 GB machine.
The machine now has two 32 GB sticks of DDR4-3200 in the X470 board, on Ubuntu 24.04. Nothing else changed in the software. On the first afternoon I did 21 starts in a row with the full config, no hangs and no GPU error events. Start time with a warm cache is about 84 s. Swap stayed at zero. I ran three more starts each for a few other variants after that, with no hangs either. I never saw it hang on 64 GB.
I’m treating this as a result and not a proof of cause. I did not run the 32 GB config again after the move, and 19 starts on the old setup is a small sample. A hang rate of 5 in 19 against 0 in 30 is hard to put down to luck, though.
One thing that did not fix it: rolling back to the older image. That arm crashed on start, and one of its GPU resets took the kernel down with it.