Introduction: The Proof, Not the Prediction
If you read Beyond the Waiting Game, you know the thesis. The memory wall is real. The GPU spends 90% of its time waiting for data, not computing. The path forward is algorithmic: find ways to generate more tokens per memory load, shift from memory-bound sequential generation to compute-bound parallel generation, and stack techniques because they address different bottlenecks.
That book was written as a survey of workarounds, organized by family. Speculative decoding here. Diffusion there. Attention compression somewhere else. The stacking chapter at the end was aspirational — a vision of what a fully optimized inference stack might look like, not a concrete product.
This book is about what happened the week after that one was finished.
NVIDIA's Nemotron-Labs-Diffusion (NLD) is not another entry in the catalog of tricks. It's the first production-scale model that embodies the thesis deliberately — trained from scratch to do both autoregressive and diffusion generation, with a self-speculation mode that stitches them together. It matches Qwen3-8B's accuracy at 3-4x the throughput on the same hardware.
But this book isn't a celebration. The interesting questions are the ones the press release glosses over:
- Why has this only been done at 3B, 8B, and 14B — not 70B or 400B?
- What's the training cost nobody mentions?
- How does the advantage change with concurrency?
- What does this mean for hardware purchasing decisions — today and next year?
- When does this paradigm break down?
The thesis from the first book was: algorithmic cleverness can postpone the memory wall indefinitely. NLD is the strongest evidence yet that this thesis is correct. But every workaround has its own limits, and those limits determine when and where the technique is useful.
This book is a map of those limits.