What This Means for Hardware

If diffusion LM inference becomes standard, the bottleneck shifts from memory bandwidth to compute density.

AR inference is memory-bound. The GPU sits idle, waiting for weights to arrive from HBM. The most important hardware metric is HBM bandwidth — more bandwidth means faster AR generation.

Diffusion inference is compute-bound. The GPU's arithmetic units are the bottleneck because generating blocks of tokens in parallel requires more matrix multiplications per forward pass. The most important hardware metric becomes FLOPS — more compute means faster diffusion generation.

This is a critical distinction for hardware purchasing:

| | AR-optimized | Diffusion-optimized |

|---|---|---|

| Bottleneck | HBM bandwidth | FLOPS |

| Key metric | GB/s per token | TFLOPS per token |

| Best hardware | H200 (high bandwidth) | B200 (high compute) |

| Edge viability | Poor (needs bandwidth) | Good (uses compute) |

For an organization buying hardware today, the question becomes: will diffusion LMs be standard within the lifetime of this hardware purchase?

If yes, you should bias toward compute density over bandwidth. The B200's 4.5 petaFLOPS of FP8 compute becomes more valuable than the H200's 4.8 TB/s of HBM bandwidth. If no — if AR models remain dominant — you should bias the other way.

The paper's data points are useful here. On a DGX Spark (edge device, modest compute), NLD gets 2.7x speedup. On a GB200 (datacenter monster), the same model gets 3.3-4x. The advantage holds across hardware tiers. This suggests diffusion is not architecture-dependent; it's a consistent improvement across the GPU lineup.