EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Recommendation System Architecture at Scale
Users
230M users
Items
15K titles
Scale
1B+ predictions/day
Final Served
10 items
Netflix Recommendation PipelineUser RequestCandidate GenerationTwo-Tower NN → 1,000 itemsTwo-Tower Embedding ModelPre-RankingGBDT / LightGBM → 100 itemsGradient Boosted TreesRankingDeep NN → 20 itemsWide & Deep / DLRMRe-RankingRules / Diversity → 10 itemsMMR + Business Rules→ Served to user (10 items)Data InfraUser Features(Online Store)Item Features(Feature Store)Logs (Kafka)
Real-time features: User features fetched from Redis/DynamoDB in <5ms. Enables capturing very recent behavior (last 5 clicks). Higher infra cost but better relevance.
Controls
Platform Style
Funnel Sizes
Candidates (Two-Tower)1,000
Pre-Rank output100
Rank output20
Re-rank output10
Options
Two-Tower: separate encoders for users and items. Dot product similarity. Cheap at inference - precompute item embeddings offline.

Re-ranking: enforces diversity (not all action movies), freshness, business rules (promotions, copyright).

Recommendation System Architecture at Scale - Interactive Visualization

Industrial recommendation systems serving billions of users cannot score every item for every user. The standard architecture uses a four-stage funnel: candidate generation (two-tower embedding model narrows millions of items to thousands), pre-ranking (gradient boosted trees score thousands to hundreds quickly), ranking (deep neural network with cross-features scores hundreds to tens with full model capacity), and re-ranking (applies diversity, freshness, and business rules to produce the final list). Each stage trades off recall for latency and compute. Netflix, Spotify, and YouTube all use variants of this architecture.

  • Two-tower model: separate user and item encoders, offline item embedding precomputation, ANN retrieval at inference
  • Pre-ranking: gradient boosted trees (LightGBM) score 1000→100 items in <5ms without GPU
  • Ranking: Wide & Deep or DLRM with user-item cross features scores 100→20 in <50ms on GPU
  • Re-ranking: Maximum Marginal Relevance (MMR) ensures diversity; business rules inject promotions and copyright
  • Real-time features: Redis/DynamoDB online store serves user features in <5ms per request
  • A/B testing layer: different ranking models serve different user buckets simultaneously for controlled experiments

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.