Three data views that make the architectural case for multi-agent systems.
The multi-agent advantage is largest on tasks with multiple distinct domains (research synthesis, complex planning). The gain is smallest on focused single-domain tasks (data analysis: +17pp vs research synthesis: +21pp). Multi-agent architecture earns its coordination overhead most on cross-domain work.
Data synthesized from AutoGen (Wu et al. 2023, arXiv:2308.08155) benchmark results and LangGraph production case studies. Task success defined as completing the stated goal to evaluator specification without human correction.
A per-step reliability of 90% sounds high. It means your agent fails one step in ten. On a 10-step task, that compounds to a 35% success rate overall. The solution is decomposition: shorter tasks per agent, fewer compounding failure points. Each worker in a multi-agent system runs a 2-4 step subtask, not a 10-step mega-task.
85% per-step reliability on a 10-step task gives you 20% success. This is the range where many production agents live: not broken (individual steps look fine in testing) but effectively useless on real-world complex tasks. The agent produces outputs with 80% confidence that are wrong 80% of the time on end-to-end tasks.
Compounding failure math is from first principles: P(n-step success) = R^n where R is per-step reliability. Per-step reliability estimates are from internal production agent monitoring and consistent with published benchmarks on agent step-level accuracy.
Hierarchical planning (orchestrator + sub-orchestrators + workers) achieves 85% success at 4.7 average LLM calls on complex multi-domain tasks. The overhead of the extra coordination layer pays back in quality when no single orchestrator can manage all workers effectively. The threshold is roughly 6+ concurrent workers — below that, flat orchestrator-worker is simpler and nearly as effective.
The 3.2 average steps number for LLM Compiler counts sequential LLM calls, not wall-clock time. Parallel execution means many LLM calls happen simultaneously. The cost in tokens (and dollars) is similar to Plan-and-Execute. The gain is latency — important for user-facing applications, less relevant for background jobs.
Success rates from LLM Compiler (Kim et al. 2023, arXiv:2312.04511) and Plan-and-Execute benchmarks on ToolBench, HotPotQA, and structured planning tasks. Hierarchical figures from internal production benchmarks on multi-domain research tasks with 5+ domains. ReAct baseline from Yao et al. (2023).