EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Vision-Language Model Architecture
Controls
Actions
Patch Size
Model
Overlays
Stats
Image tokens256
Text tokens5
Token budget261 / 4096
Model dim4096
ViT divides the image into patches, encodes each as an embedding, then a projection layer maps them to LLM token dimension - so the LLM sees image and text as one token stream.

Vision-Language Model Architecture - Interactive Visualization

Vision-language models (VLMs) process images by dividing them into patches (typically 14x14), encoding each patch with a ViT, projecting patch embeddings into the LLM token space, and attending over both visual and text tokens jointly. A 336x336 image at patch_size=14 produces 576 visual tokens that compete for context window space alongside text.

  • Patch extraction: 336x336 image divided into 576 non-overlapping 14x14 patches
  • ViT encoding: each patch embedded into a 768-dim vector by the vision transformer
  • Projection layer: linear transformation maps visual embeddings into the LLM token dimension
  • Joint attention: visual and text tokens attend to each other across all transformer layers

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.