Speculative Decoding - Interactive Visualization

Speculative decoding accelerates LLM inference by using a small fast draft model to propose k tokens, which the large target model verifies in a single parallel forward pass. Accepted tokens are free; rejected tokens fall back to the target model's distribution. At high acceptance rates, this yields 2–3× speedup with identical output distribution.

• See the draft-then-verify loop: draft model proposes k tokens, target model scores all k in one pass
• Acceptance rate visualization - which draft tokens are accepted vs rejected by the target model
• Speedup calculator - see how acceptance rate and draft model speed translate to wall-clock speedup
• Understand why rejected tokens still produce a valid sample from the target distribution (no quality loss)
• Compare speculative decoding vs standard autoregressive decoding latency on the same sequence

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.