14 notebooks, 12 benchmarks, 3 frameworks. Cumulative standings after each week.
Week 1
#1–#2 · Setup & Environment
Install overhead, cold start, basic index time
Winner: SynapseKit
Week 1
#3 · Basic Retrieval
Cosine similarity retrieval LoC + latency
Winner: SynapseKit
Week 1
#4 · Reranking
Cross-encoder reranking pipeline LoC
Winner: SynapseKit
Week 1
#5–#6 · Eval Harness
Built-in evaluation + RAGAS integration
Winner: LangChain
Week 1
#7 · Week 1 Scorecard
Aggregate: 6 benchmarks, 3-2-1 points
Final: SK 15 · LI 8 · LC 7
Week 2
#8 · RAG from PDF
Total LoC: imports + functional code
Winner: SynapseKit — 7 vs 13 vs 11 lines
Week 2
#9 · Chunking Strategies
Built-in splitter count + configurability
Winner: LlamaIndex — 9 vs 7 vs 4 splitters
Week 2
#10 · Built-in BM25
Extra packages required for BM25
Winner: SynapseKit — 0 extra packages
Week 2
#11 · Hybrid Search RRF
RRF configurability score (out of 5)
Winner: LangChain — 5/5 configurability
Week 2
#12 · Streaming TTFT
Median framework overhead to first token
Winner: SynapseKit — 0.001ms overhead
Week 2
#13 · Conversation Memory
LoC to add multi-turn memory to RAG
Winner: SynapseKit — 4 vs 12 vs 6 lines