Afterword: The Next Unit of Progress

Every technique described in this book — every algorithmic workaround, every hardware architecture, every economic insight — is a response to one physical fact. The speed of light is finite. Memory is far from compute. And no one is going to change either.

What changes is how we work within the constraint. The techniques in this book were developed over the last three to five years — an astonishing pace given the complexity of the hardware and software involved. FlashAttention, MEDUSA, EAGLE, DeepSeek MoE, Groq LPU, Cerebras wafer-scale. Each one chips away at a different face of the memory wall.

None of them is the final answer. The pace of innovation is accelerating. Each breakthrough reveals the next layer of the constraint. The engineers who understand the physics — the people who can look at a roofline chart and see not an obstacle but a map — will find the next unit of progress.

The waiting game is not a lament. It is a description of terrain. The people who learn to read the terrain will go faster than the people who pretend it does not exist.