AI Letters #13 - The RAM Cost of LLM Frameworks. Measured.

March 29, 2026 · 7 min read

AI Engineering Education

The framework doesn't own your memory budget. The embedding model does. Pick your framework on other criteria.

Your RAG app is using 380 MB. First instinct: blame the framework. Switch to something leaner. Maybe that'll bring it down to 200. It won't - the 380 MB isn't the framework. It's the embedding model sitting in RAM. The framework overhead, across the full lifecycle from import to active use, accounts for 20–30 MB of difference between your best and worst options. You were optimizing the wrong variable.

The number that actually matters is the import floor - what you pay before any LLM work starts, before you index a single document, before the embedding model loads. I measured RSS at three stages across SynapseKit, LangChain, and LlamaIndex to find it.

Disclosure: I'm the author of SynapseKit. All benchmarks are reproducible - the Kaggle notebook is public: LLM Showdown #4 - Memory Footprint.

What We Measured

Three stages for each framework, measured with psutil.Process().memory_info().rss:

Stage	What it captures
Import	Floor cost - just having the framework in process
After indexing 200 docs	Memory after embedding + storing a real corpus
After 50 queries	Memory after active retrieval - does it accumulate?

Plus: how does RAM scale as corpus size grows from 10 to 400 documents?

Same embedding model for all three: all-MiniLM-L6-v2 via sentence-transformers. Same 200-document corpus. Same 5 queries, run 10 times each.

The Numbers

Framework	Import	After Index	After Queries	Total Growth
SynapseKit	89 MB	193 MB	196 MB	+107 MB
LangChain	82 MB	207 MB	210 MB	+128 MB
LlamaIndex	114 MB	218 MB	221 MB	+107 MB

What the Numbers Say

LangChain has the lowest import floor - by a margin.

This is counterintuitive. LangChain ships 32+ packages at full install. LlamaIndex ships 60+. Yet LangChain's import RSS is the lowest. Reason: lazy loading. LangChain defers loading provider integrations until first use. LlamaIndex loads more eagerly - node parsers, document stores, and pipeline infrastructure initialise at import.

After indexing, the embedding model dominates everything.

The jump from import to after-index (104–125 MB across frameworks) is almost entirely all-MiniLM-L6-v2 loading into memory (~90 MB). Framework overhead on top of that is small. All three end up within 25 MB of each other despite a 32 MB gap at import.

After 50 queries, memory barely moves.

All three show +2–4 MB of growth after 50 queries. Retrieval doesn't accumulate. Once the index is built, query cost is compute, not memory.

The Scaling Story

Memory growth vs corpus size (docs indexed)
─────────────────────────────────────────────────────────
         10     25     50    100    200    400 docs
────────────────────────────────────────────────────────
SK      ~99   ~103   ~109   ~125   ~193   ~199 MB
LC      ~97   ~102   ~111   ~133   ~207   ~215 MB
LI     ~122   ~126   ~133   ~151   ~218   ~226 MB
─────────────────────────────────────────────────────────
Slope:  nearly identical - vector math determines this
Vector: 384 dims × 4 bytes × ~3 chunks/doc ≈ 4.6 KB/doc

The slopes are nearly parallel. At 400 documents, the gap between best and worst is 27 MB. Memory growth is determined by vector storage, not framework architecture.

What This Means for Engineers

In serverless/edge: the import floor is the only number that matters. A Lambda with a 256 MB limit doesn't care whether your framework peaks at 207 MB or 218 MB. It cares whether you breach 256 MB before indexing starts. LlamaIndex costs 32 MB more than LangChain at import - that's the margin between working and OOM in constrained environments.
Optimizing framework choice for memory is the wrong lever. Smaller embedding model, smaller chunks, quantized index - those are the right levers. Framework swap saves 10–20 MB. Switching from local embeddings to API calls saves 90 MB.
More packages ≠ more runtime memory. Package count is an install-time concern. Runtime memory is a loading strategy concern. LangChain's lazy architecture decouples these completely.
Memory doesn't accumulate with query load. If your service is growing in memory under traffic, the culprit is application-level caching or a reference leak - not the framework.
The embedding model is your first memory optimization target. Moving to API-based embeddings eliminates ~90 MB of in-process RAM. In RAM-constrained environments that tradeoff is often worth it.

The Thing Most People Miss

The 25 MB spread between frameworks at peak is real. But at 10,000 documents, vector data alone is ~45 MB. At 100,000 documents, it's ~450 MB. At that scale you're thinking about ANN indexes and external vector databases - not framework choice. The memory benchmark matters most in constrained environments: serverless, edge, Lambda. In that regime, LangChain's lazy loading is a genuine advantage. Everywhere else, optimize the embedding model first.

Three Things Worth Doing This Week

Profile your import floor. Run python -c "import your_rag_module; import psutil, os; print(psutil.Process(os.getpid()).memory_info().rss/1e6, 'MB')" before your app does any work. That's the number your Lambda sees on cold start.
Separate embedding model memory from framework memory in your metrics. The jump from import to post-embed is your model cost. Everything else is framework overhead. Knowing which is which tells you which lever to pull.
If you're in serverless, measure in your actual runtime. The Kaggle environment has numpy/pandas pre-loaded. Your Lambda doesn't. Fork the notebook, adapt the setup, measure what actually matters for your constraints.

The memory benchmark doesn't tell you which framework is better. It tells you what each framework costs before your actual work begins. That's a different - and more actionable - question.

Engineers of AI

Read more: www.engineersofai.com

If this was useful, forward it to one engineer who should be reading it.