What Self-Speculation Actually Is

The term "self-speculation" matters because it's the cleanest synthesis yet of the two threads from the first book.

Traditional speculative decoding (Chapter 1 of Beyond) requires a draft model — either a separate small model, attached heads (MEDUSA), or feature-predicting heads (EAGLE). Every variant adds complexity: training the draft model, keeping it in sync, managing the overhead.

Self-speculation eliminates all of that. The same model drafts (in diffusion mode) and verifies (in AR mode) using the same weights. The draft isn't produced by a separate system; it's a different behavior of the same neural network, enabled by changing how it reads its own attention patterns.

The shared KV cache is what makes this efficient. When the diffusion mode computes a draft block, the intermediate representations are already in the cache. The AR verification pass reuses them — it doesn't recompute anything from scratch. The cost of verification is approximately one forward pass, regardless of how many tokens are in the draft.

This is why self-speculation achieves higher acceptance lengths than MTP methods. EAGLE-3 accepts around 6 tokens per verification on average. NLD's self-speculation accepts more — the paper reports 3x higher acceptance length than EAGLE-3, implying roughly 18 tokens per verification. That's 18 tokens generated from a single full model load, versus AR's 1 token per load.