The Custom Kernel Dependency
The peak numbers (4x, 1,015 tok/s) require custom CUDA kernels. The "stock" numbers (3.3x, 850 tok/s) use standard SGLang + Flash Attention.
This is a pattern that will repeat. The theoretical ceiling of any new inference technique is always higher than what standard frameworks deliver. Custom kernels for NLD's unique attention patterns — the clean-stream design, the block-wise bidirectional/causal masks — can unlock additional throughput.
The implication: production adoption will be gated not just on the model quality but on framework support. SGLang already has a pending merge request for NLD support. But TensorRT-LLM, vLLM, TGI, and the other serving frameworks will need to implement their own kernel optimizations. This takes months.
Until the framework ecosystem catches up, NLD's performance will vary significantly by deployment stack. The paper's numbers are reproducible with SGLang, but the custom-kernel numbers are a future state.