AI Letters #13 · Memory Footprint

RAM Evidence Dashboard

All the numbers from LLM Showdown #4. Stage comparison, corpus scaling curves, and a decision guide for what to optimize based on your constraints.

82 MB

LangChain import (lowest)

114 MB

LlamaIndex import (highest)

~90 MB

Embedding model cost

~25 MB

Max spread at peak

Stage comparison — RSS (MB)

Memory growth (import → after 50 queries)

Memory scaling — corpus size (docs indexed)

Full benchmark table

Framework	Import (MB)	After Index (MB)	After Queries (MB)	Index Growth	Query Growth	At 400 docs
SynapseKit	89	193	196	+104 MB	+3 MB	~199 MB
LangChain	82 ✓	207	210	+125 MB	+3 MB	~215 MB
LlamaIndex	114	218	221	+104 MB	+3 MB	~226 MB
Embedding model (all-MiniLM-L6-v2): ~90 MB — accounts for most of the import→index jump across all frameworks. Environment: Kaggle CPU. Reproducible: LLM Showdown #4

When does memory actually matter — decision guide

Serverless / Lambda / edge functions

Import floor is everything. LangChain's 82 MB vs LlamaIndex's 114 MB is a 32 MB difference before any work starts. In a 256 MB Lambda, that 32 MB is the margin between working and OOM. Prefer lazy-loading frameworks or pre-warm aggressively.

Long-running services (API servers, containers)

Peak memory converges. After loading the embedding model, all three frameworks sit within 25 MB of each other. At this scale, optimize the embedding model (API vs local) not the framework. Pick on latency and DX criteria.

Large corpus (thousands of documents)

Framework choice barely matters. Vector storage dominates: 384 dims × 4 bytes × ~3 chunks/doc ≈ 4.6 KB/doc. At 10,000 docs that's ~45 MB of vectors regardless of framework. Switch to an external vector DB before worrying about framework overhead.

Memory is growing with query traffic

Not the framework. Retrieval doesn't accumulate memory (+3 MB after 50 queries across all frameworks). Memory growth under traffic is a result caching, reference leak, or GC issue in your application code — not framework behavior.