What NVIDIA Actually Built

Let's be precise about what NLD is and isn't.

NLD is a family of language models (3B, 8B, 14B, plus an 8B VLM) trained on a Ministral3 base with a specific two-stage recipe:

• Stage 1: 1 trillion tokens of standard autoregressive continued pretraining
• Stage 2: 300 billion tokens of joint autoregressive + block diffusion training

The joint training is the key innovation. The model learns two objectives simultaneously: predict the next token (standard AR loss) and denoise a masked block of tokens (diffusion loss). These objectives share the same parameters, computed in a single forward pass via a carefully designed attention mask that prevents label leakage.

After training, the model can run in three modes, switched by changing the attention pattern at inference time:

Mode 1 — Autoregressive (AR). Standard next-token prediction, one token at a time, causal attention. This is what every LLM does. The baseline. 41.8 tok/s on a DGX Spark.

Mode 2 — Diffusion (dLM). Generate a block of tokens in parallel from noise. The model takes a masked sequence (all tokens hidden) and iteratively denoises the block — typically 32 tokens — refining it over multiple steps. 112 tok/s on a DGX Spark.

Mode 3 — Self-Speculation. Diffusion drafts a block. AR verifies the entire block in a single forward pass. Tokens the AR head accepts are output immediately; any rejection resets to the accepted prefix and the diffusion drafts again. 850 tok/s on a GB200.

The numbers are the headline. On a GB200, NLD-8B in self-speculation mode generates 850 tokens per second — 3.3x faster than the same model in AR mode (253 tok/s), and 2.4x faster than Qwen3-8B running Eagle3 (360 tok/s). With custom CUDA kernels, they hit 1,015 tok/s — 4x AR.

The DGX Spark numbers are more interesting for the long tail: 112 tok/s on an edge device. That's not just fast for edge inference; it's fast enough for real-time conversation on a device that fits on a desk.