DMA

Some driver bugs feel almost random.

The CPU has written a descriptor, but the device reads old contents. The DMA completion interrupt has fired, but the driver still reads stale buffer data. Adding one log line makes the bug disappear. Changing optimization brings it back. It worked on a single-core MCU, then fails occasionally on an SoC with cache.

These bugs are often not caused by “broken DMA” or “an aggressive compiler.” They happen because several different guarantees were mixed together.

A common DMA driver bug looks like this: the CPU prepared a buffer, but the device reads old data; the device finished DMA, but the CPU still sees old values; the driver works on one SoC but fails after enabling IOMMU.

The usual root cause is treating a CPU pointer as an address the device can use.

In Linux, user virtual addresses, kernel virtual addresses, physical addresses, and DMA addresses are different layers. A device should receive a DMA address, not an ordinary C pointer.

One of the most confusing driver bugs is “the data is clearly in memory, but the other side cannot see it.”

The CPU prepares a transmit buffer, but the network device sends old data. DMA has written received data into memory, but the driver or application still reads stale values. Adding logs makes the issue disappear; changing optimization brings it back.

These bugs are often not because DMA failed or the pointer is wrong. The problem is that CPU cache, DMA device, and memory visibility were not handled correctly.