EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Back-of-Envelope Estimation for ML Systems
Back-of-Envelope Estimator
Recommendation - Capacity Planning
Avg QPS
579 req/s
Peak QPS (3×)
2K req/s
Storage / day
102.40 GB
Storage / month
3.07 TB
Bandwidth
0.03 Gbps
GPUs needed
174
Monthly cost
$438.5K
Feature store
20.48 GB
QPS Breakdown
Avg QPS579 req/s
Peak QPS (3×)2K req/s
After cache (40%)1K req/s
Calculation Steps
# Step 1: Requests per day
10.0M DAU × 5 req/user = 50.0M req/day
# Step 2: Average QPS
50.0M / 86,400 sec = 578.7 QPS
# Step 3: Peak QPS
578.7 × 3 = 1.7K peak QPS
# Step 4: GPU count
1 GPU handles 10 req/s @ 100ms → 174 GPUs
# Step 5: Storage/day
50.0M req × 2048 B = 102.40 GB
# Step 6: Monthly cost
174 GPUs × $3.50/hr × 720 hr = $438.5K/mo
System Type
Inputs
DAU10.0M
Req / user / day5
Model size500MB
Inference latency100ms
Feature vec size2048B
Display
Rule of Thumb
Peak QPS = avg × 3–5×. Always design for peak, not average. Caching 40% of requests reduces effective load significantly.

Back-of-Envelope Estimation for ML Systems - Interactive Visualization

Back-of-envelope estimation is the first skill tested in ML system design interviews. Given daily active users, requests per user, model size, and inference latency, you can derive average QPS, peak QPS (3× average), storage per day, bandwidth, GPU count needed, and monthly cloud cost. These numbers determine whether your architecture is feasible before writing a single line of code. The rule: always design for peak, not average load.

  • Average QPS = DAU × requests/user ÷ 86,400 seconds per day
  • Peak QPS = average × 3–5× - use 3× for most systems, 5× for event-driven products
  • GPU count = ceil(peak QPS ÷ (1000ms ÷ inference latency)) - each GPU handles N requests/sec
  • Monthly cost scales linearly with GPU count - quantization and caching are the highest-leverage optimizations

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.