EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Outlines - Grammar FSM for Structured LLM Output
JSON Grammar
object = {
key-value-pairs
}
key = string
value = string | number
Example: {"name": "Alice", "age": 30}
Generated Output (step 0/9)
?
FSM state: Start
Finite State Machine - Current State Highlighted
Start
→
In obj
→
Key
→
Colon
→
Value
→
Done
Token Mask - Valid (green) vs Masked (gray)Next: "{"
{
}
"name"
"age"
"city"
:
"Alice"
"Bob"
42
30
,
1
2
3
4
5
-
a
b
c
A
B
C
<EOS>
Controls
Grammar Type
JSON
regex
EBNF
Outlines: Compiles grammar to an FSM, then masks logits at each step so only grammatically valid tokens can be sampled. Zero probability to invalid tokens - guaranteed structured output.

Outlines - Grammar FSM for Structured LLM Output - Interactive Visualization

Outlines converts a grammar (JSON schema, regex, or EBNF) into a finite state machine (FSM). At each generation step, it looks up which FSM transitions are valid from the current state, then sets the probability of all invalid tokens to zero. Only tokens that would keep the FSM in a valid state can be sampled. This guarantees grammatically valid output without any retry logic or post-processing.

  • Compile grammar to FSM once - then apply at inference time with near-zero overhead
  • At each step: find current FSM state, look up valid next tokens, mask all invalid tokens
  • Guarantees 100% valid JSON, regex match, or EBNF-conformant output - no retries needed
  • Works with any huggingface model - adds a logit processor to the generation pipeline
  • Supports JSON Schema, Pydantic models, regex patterns, and custom EBNF grammars

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.