EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Structured Output and Constrained Decoding
JSON Schema
{
  "type": "object",
  "properties": {
    "name": {
      "type": "string"
    },
    "age": {
      "type": "integer",
      "minimum": 0
    },
    "email": {
      "type": "string",
      "format": "email"
    }
  },
  "required": [
    "name",
    "age"
  ]
}
Generated output
Generation will appear here…
0/13 tokens generated
Constrained decoding - next token candidates
{VALID
"Hello there!"BLOCKED
42abcBLOCKED
undefinedBLOCKED
"maybe"BLOCKED
Schema forces this token. Only "{" is structurally valid here.
Token stream
{
"name"
:
"Alice"
,
"age"
:
30
,
"email"
:
}
Schema-forced
Model-chosen (valid)
Current step
Controls
Schema
Temperature0.7
Affects model-chosen tokens only. Forced tokens ignore temperature.
Constrained decoding uses a Finite State Machine (FSM) from the JSON schema. At each step, only tokens that keep the FSM in a valid state are allowed. Invalid tokens get logit = -∞.
Tools: Outlines, Guidance, llama.cpp grammar, OpenAI response_format, Instructor library.

Structured Output and Constrained Decoding - Interactive Visualization

Structured output generation uses constrained decoding to guarantee that LLM outputs conform to a specified schema. At each decoding step, a Finite State Machine (FSM) compiled from the JSON schema determines which tokens are structurally valid. Invalid tokens receive a logit of negative infinity, making them impossible to sample. This enables reliable extraction of structured data without post-processing or retries.

  • Token-by-token animation showing valid tokens (green) vs schema-blocked tokens (red) at each generation step
  • JSON schema display for three object types: Person, Product, and Event
  • Schema-forced vs model-chosen token distinction - some tokens are uniquely determined by the schema
  • Temperature slider showing that forced tokens are unaffected by temperature (logit masking overrides sampling)

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.