AI Letters #20 - Hybrid Search: RRF Fusion Across Three Frameworks

Pure vector search misses exact matches. Pure BM25 misses semantics. Hybrid search almost always wins - the question is how much control you get over the fusion.

Every production RAG system eventually hits the same wall. Vector search retrieves semantically similar documents, but it fails on exact-match queries: model names, version numbers, function names, error codes. The query "GPT-4o" and the document "GPT-4o" don't reliably produce close vectors. BM25 doesn't have this problem. It matches terms, weighs them by rarity, and returns the right document.

Reciprocal Rank Fusion - RRF - is the standard way to combine both. It takes two ranked lists, assigns each document a score of 1 / (k + rank), sums the scores, and re-ranks. The parameter k controls how much the top ranks dominate. It requires no score normalisation, works across retrieval algorithms with incompatible score scales, and runs in microseconds.

We built identical hybrid pipelines across SynapseKit 1.4, LangChain 1.2, and LlamaIndex Core 0.14. Same corpus, same query, same task: BM25 + vector, top-3 via RRF. The LoC gap is smaller than the BM25-only benchmark. The configurability gap is not.

Interactive Chart

LoC Across the Series →

Lines of code per framework across all 11 benchmarks - from hello world to hybrid search. See which framework has compounded its lead.

Code Explorer

Hybrid Pipeline Code by Framework →

Full BM25 + vector + RRF pipeline code side by side - imports, setup, and retrieval call annotated for each framework.

Data

LoC, RRF Configurability, and Result Overlap →

Lines of code breakdown, configurable RRF parameters per framework, and result overlap across frameworks on an identical hybrid query.

What We Measured

Task: Index 5 documents with both BM25 and vector search, run an identical query through each hybrid retriever, return top-3 results via RRF fusion.

Metric	What it captures
Lines of code	Code to build and query a hybrid BM25 + vector pipeline
RRF configurability	Parameters exposed: weights, k, retriever count
Result agreement	Overlap in top-3 results across frameworks

Frameworks: SynapseKit 1.4, LangChain 1.2, LlamaIndex Core 0.14. Kaggle CPU. Disclosure: I'm the author of SynapseKit. All code is on Kaggle - fork and run yourself.

The Code

SynapseKit - 8 lines (2 imports + 6 functional):

from synapsekit.retrieval import HybridSearchRetriever, Retriever, InMemoryVectorStore
from synapsekit.embeddings import SynapsekitEmbeddings

emb    = SynapsekitEmbeddings(model="all-MiniLM-L6-v2", use_gpu=False)
r      = Retriever(InMemoryVectorStore(emb))
hybrid = HybridSearchRetriever(r, bm25_weight=0.5, vector_weight=0.5, rrf_k=60)
hybrid.add_documents(DOCS)
await r.add(DOCS)
results = await hybrid.retrieve(QUERY, top_k=3)

A single HybridSearchRetriever class wraps both modes. bm25_weight, vector_weight, and rrf_k are explicit constructor parameters. Limitation: fixed at two retrievers.

LangChain - 11 lines (4 imports + 7 functional):

from langchain_classic.retrievers.ensemble import EnsembleRetriever
from langchain_community.retrievers import BM25Retriever
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_community.embeddings import HuggingFaceEmbeddings

emb    = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
vs     = InMemoryVectorStore(emb)
vs.add_texts(DOCS)
bm25   = BM25Retriever.from_texts(DOCS, k=3)
vec_r  = vs.as_retriever(search_kwargs={"k": 3})
hybrid = EnsembleRetriever(retrievers=[bm25, vec_r], weights=[0.5, 0.5])
results = [doc.page_content for doc in hybrid.invoke(QUERY)]

EnsembleRetriever is compositional: pass a list of any retrievers, a matching weights list. Add a third retriever by appending to both lists.

LlamaIndex - 12 lines (4 imports + 8 functional):

from llama_index.core.retrievers import QueryFusionRetriever, VectorIndexRetriever
from llama_index.core import Document, VectorStoreIndex, Settings
from llama_index.core.node_parser import SentenceSplitter
from llama_index.retrievers.bm25 import BM25Retriever

Settings.llm = None
nodes  = SentenceSplitter(chunk_size=512).get_nodes_from_documents(
             [Document(text=d) for d in DOCS])
bm25_r = BM25Retriever.from_defaults(nodes=nodes, similarity_top_k=3)
vec_r  = VectorIndexRetriever(index=VectorStoreIndex(nodes), similarity_top_k=3)
fused  = QueryFusionRetriever([bm25_r, vec_r], similarity_top_k=3,
                              num_queries=1, use_async=False)
results = [n.text for n in fused.retrieve(QUERY)]

Twelve lines. Node parsing is unavoidable LlamaIndex boilerplate. The RRF k parameter is fixed internally and not exposed.

The Numbers

Framework    Imports   Functional   Total
──────────────────────────────────────────
SynapseKit       2           6         8
LangChain        4           7        11
LlamaIndex       4           8        12

The gap is smaller here than in BM25-only (where LangChain won at 3 lines). Hybrid search adds enough setup that the difference compresses. Four lines separate the most concise from the most verbose.

RRF Configurability

Parameter             SynapseKit   LangChain    LlamaIndex
────────────────────────────────────────────────────────────
BM25 weight           Yes          Yes          No
Vector weight         Yes          Yes          No
RRF k constant        Yes          Yes          No
Retriever count       2 only       Unlimited    Unlimited
Async support         Yes          Yes          Yes

Configurability       4/5          5/5          3/5

LlamaIndex's QueryFusionRetriever applies equal weighting to all retrievers. There is no weights parameter. If BM25 produces more false positives than vector, you cannot correct for it.

SynapseKit exposes weights and the k constant explicitly. The tradeoff: fixed at two retrievers. You cannot add a sparse retriever or reranker as a third leg.

LangChain is the most flexible. EnsembleRetriever takes weights=[0.3, 0.5, 0.2] for three retrievers. You can mix BM25 + dense + sparse + reranker in one call and tune the contribution of each signal.

Result Overlap

Query: "How does hybrid search combine BM25 and vector retrieval?"

Rank   SynapseKit                     LangChain                      LlamaIndex
────────────────────────────────────────────────────────────────────────────────
#1     Vector search uses dense...    TF-IDF and BM25 both use...    Hybrid search combines...
#2     Hybrid search combines...      Vector search uses dense...     Vector search uses dense...
#3     BM25 is a probabilistic...     Hybrid search combines...       TF-IDF and BM25 both use...

Jaccard: LangChain vs SynapseKit 0.75  |  LangChain vs LlamaIndex 0.75  |  LlamaIndex vs SynapseKit 0.50

What This Means for Engineers

1. LangChain's EnsembleRetriever is the right default for production hybrid search. Unlimited retriever composition with per-retriever weights is what you want when you're tuning a real pipeline. The extra 3 lines over SynapseKit are worth it.

2. LlamaIndex's no-weight limitation is a real constraint. Equal-weighting RRF works as a starting point. It fails when one retrieval mode dominates false positives and you need to downweight it.

3. SynapseKit's single-class API is convenient for the 2-retriever case. If you're doing standard BM25 + dense and never need a third leg, the explicit bm25_weight, vector_weight, rrf_k API is clean.

4. RRF k=60 is not magic. Lower k amplifies the importance of rank-1 results. Higher k flattens the distribution. Experiment with k in the range 30–100 before assuming 60 is optimal.

5. Hybrid search is not free. You're running two retrieval steps plus a merge. Use asyncio.gather() to run BM25 and vector concurrently - LangChain supports ainvoke() on all its retrievers.

The Corollary Most People Miss

Hybrid search architecture choice:
──────────────────────────────────────────────────────────
SynapseKit    Single class, explicit weights, fixed at 2
              + clearest API for standard hybrid
              - cannot extend to 3+ retrieval signals

LangChain     Composable list, per-retriever weights
              + most flexible for production tuning
              - 3 more lines, more imports to manage

LlamaIndex    Composable list, equal weights only
              + supports unlimited retrievers
              - no weight control - blind spot for prod

Three Things Worth Doing This Week

1. Add asyncio.gather() to your hybrid retriever. If you're running BM25 and vector sequentially, you're paying both latencies. Run them concurrently and your hybrid latency drops to the slower of the two, not the sum.

2. A/B test RRF k. Change k from 60 to 30 on a sample of your production queries. Lower k amplifies top-rank signals. Measure precision@3 on both.

3. Read the Kaggle notebook. Full reproducible code, live RRF computation, and result overlap tables: LLM Showdown #11 - Hybrid Search

---

Hybrid search is the standard, not the exception, for production RAG. The frameworks all implement RRF. What they disagree on is how much of the fusion parameters they expose to you. One treats the weights as fixed. One gives you two weights and a k constant. One gives you a weight list as long as your retriever list. That last one is the one you want when you're optimising recall across different query types.

Engineers of AI

Read more: www.engineersofai.com

If this was useful, forward it to one engineer who should be reading it.