EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Quantisation Effects on Weights
Max Error
0.00000
Mean Abs Error
0.00000
Size Reduction
4×
Bytes / Weight
1.00
negativenear zeropositive
Quantisation
Precision
Weight distribution
int8 reduces memory 4× with minimal accuracy loss. int4 reduces 8× but introduces visible rounding error on outlier weights.

4-bit quantization (GPTQ, GGUF) is the standard for serving LLMs on consumer hardware.

Watch heavy-tail distributions - outlier weights force a wide quantization range, amplifying errors on small weights.

Quantisation Effects on Weights - Interactive Visualization

Quantization reduces model size by representing weights in fewer bits - float32 uses 4 bytes per weight, int8 uses 1 byte (4x compression), int4 uses 0.5 bytes (8x compression). The tradeoff is rounding error: fewer bits means coarser grid steps and less precision. This visualization shows the weight distribution histograms and the visible quantization steps at different bit depths.

  • Float32: smooth continuous distribution of weight values, 4 bytes per weight
  • Int8: 256 discrete levels, staircase pattern visible in histogram, minimal accuracy loss
  • Int4: 16 levels, significant rounding error on outlier weights, ~0.5% accuracy drop
  • Int2: 4 levels - dramatic rounding visible, generally not usable for LLMs
  • 8x8 weight matrix grid: see float vs quantized values side-by-side with color coding
  • Why this matters: running LLaMA 3 70B on consumer hardware requires 4-bit (GGUF/GPTQ)

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.