PDF RAG — Lines of Code & Indexing Time
Benchmark #8: PDF file → queryable index. Measured on Kaggle CPU, all-MiniLM-L6-v2 embeddings, median of 3 runs.
SynapseKit
4
1 import + 3 functional
3.1 s indexing
LlamaIndex
8
3 imports + 5 functional
3.4 s indexing
LangChain
14
5 imports + 9 functional
2.8 s indexing
Lines of Code — Imports vs Functional
Stacked bar. Lower is simpler API surface.
Indexing Time (PDF → searchable index)
Median of 3 runs. Kaggle CPU, all-MiniLM-L6-v2.
What Changed vs Hello RAG (#3 — string input)
| Framework |
#3 (string) LoC |
#8 (PDF) LoC |
Delta |
What changed |
| SynapseKit |
4 |
4 |
0 |
Nothing. rag.add() is format-agnostic. |
| LangChain |
13 |
14 |
+1 |
Added PyPDFLoader import; removed Document() wrap. |
| LlamaIndex |
9 |
8 |
−1 |
SimpleDirectoryReader replaced Document(text=...) wrap. |
Detailed Line Count Breakdown
| Framework |
Import lines |
Functional lines |
Total |
PDF loader type |
| SynapseKit |
1 |
3 |
4 |
Automatic (no explicit loader) |
| LlamaIndex |
3 |
5 |
8 |
SimpleDirectoryReader (auto-detects) |
| LangChain |
5 |
9 |
14 |
PyPDFLoader (1 of ~50 loaders) |
On indexing time
All three are within 21% of each other (2.8 s – 3.4 s). The dominant cost is the embedding model: all-MiniLM-L6-v2 encodes the same chunks regardless of which framework calls it. Framework overhead — PDF parsing, object construction, internal validation — is real but secondary. Do not choose your framework based on indexing time for a 3-page PDF.
LlamaIndex honest nuance
SimpleDirectoryReader auto-detects file types from extension. Pass it a directory with PDFs, Word docs, HTML, and plain text — it routes each file to the correct parser without you specifying the type. That's genuinely useful for mixed-format corpora. The line count advantage over LangChain is modest; the capability advantage is real.
LangChain honest nuance
PyPDFLoader is one of ~50 document loaders in langchain-community — Confluence, Notion, GitHub, SharePoint, S3, and more. All return List[Document] with the same .load() interface. The verbosity buys you a consistent loading API across every data source you'll ever need. Each chunk also preserves page-level metadata (page number, source path) that's invisible in SynapseKit's abstraction.
www.engineersofai.com · AI Letters #17