25%
RAG accuracy gap between best and worst embedding model (MTEB)
30%
Accuracy drop when answer is in middle of context (Liu et al. 2023)
<1%
Parameters updated in LoRA — same performance as full fine-tuning
Evidence #1 — Embeddings
Embedding Model Choice Determines RAG Accuracy
Source: HuggingFace MTEB Leaderboard — Retrieval benchmark (nDCG@10)
What this means: The gap between the best and worst embedding model on the same retrieval task is 15–25 nDCG@10 points. That's not a marginal difference. A RAG system built on the wrong embedding model can fail in production not because the architecture is wrong, but because nobody compared embedding models on the actual data distribution. This is a skills problem, not a tools problem.
Evidence #2 — Attention
"Lost in the Middle" — Attention Doesn't Distribute Evenly
Source: Liu et al. (2023) "Lost in the Middle: How Language Models Use Long Contexts" — Multi-document QA accuracy
What this means: LLMs systematically underperform when relevant information appears in the middle of a long context window. This is a consequence of how attention score distributions behave — not a bug, an architectural property. Engineers who understand attention design retrieval pipelines that put the most relevant chunks first and last. Engineers who don't wonder why their long-context RAG is failing.
Evidence #3 — Fine-Tuning
LoRA: <1% of Parameters, Full Fine-Tuning Performance
Source: Hu et al. (2021) "LoRA: Low-Rank Adaptation of Large Language Models" — GPT-3 on E2E NLG benchmark
Parameter efficiency
Performance comparison
What this means: LoRA matches full fine-tuning on most tasks while updating fewer than 1% of model parameters — which means 100x lower GPU memory cost. This made domain adaptation economically viable for companies that can't afford to fine-tune 70B parameter models. The bottleneck moved from compute to data quality and evaluation methodology. That's the skill gap most teams haven't closed.