EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Edge ML Deployment
Three Deployment Tiers
☁
Cloud
Model: 1000 MB  |  Latency: ~200ms  |  Accuracy: 99%
use case
Complex reasoning
🖥
Edge Server
Model: 120 MB  |  Latency: ~30ms  |  Accuracy: 95%
use case
Batch processing
📱
On-Device
Model: 100 MB (compressed)  |  Latency: ~3ms  |  Accuracy: ~86.5%
use case
Wake word / offline
Model Compression Pipeline
Original 400 MB
→
Quantization
→
Pruning
→
Distillation
→
100 MB ONNX/TFLite Export
Size reduction75% smaller (100 MB)
Accuracy retained98.5% (within tolerance)
Device Fit: Mobile (iPhone/Android)
Max model size
300 MB
Compressed model
100 MB ✓ fits
RAM: 4–8 GB  |  Compute: A-series NPU / Snapdragon
Bandwidth Saved (model update)
Full model (cloud sync)400 MB
Compressed (OTA update)100 MB
Compression Controls
Technique
Accuracy Tolerance
max loss3%
0%10%
Target Device
Display
Edge ML trades model size and accuracy for latency and privacy.

A 400 MB model becomes 4 MB after full compression - small enough to ship OTA to millions of devices.

Edge ML Deployment - Interactive Visualization

Edge ML pushes model inference to the device rather than the cloud - eliminating network round-trips, improving privacy, and enabling offline use cases. The pipeline starts with a 400 MB cloud model and applies compression techniques: INT8 quantization reduces size by 75% with ~1.5% accuracy loss; weight pruning removes redundant connections; knowledge distillation trains a smaller student model to mimic a larger teacher. The final compressed model is exported to ONNX or TFLite for on-device runtime. The key constraint is whether the compressed model fits within the target device's available memory.

  • INT8 quantization: replaces 32-bit float weights with 8-bit integers - 4x size reduction, minimal accuracy loss
  • Weight pruning: zeroes out low-magnitude weights (60% sparse) - reduces compute, requires sparse runtime support
  • Knowledge distillation: trains a small student model on the outputs of a large teacher - the most aggressive size reduction
  • MCUs have 256 KB to 2 MB of RAM - only tiny models (TensorFlow Lite Micro) can run here

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.