EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Agent Evaluation - Trajectory and Error Analysis
Completion Rate
44/50
87%
Avg Steps
3.4
optimal: 2
Efficiency
0.59
steps ratio
Success Rate
87%
50 runs
Step Scores
1. Parse question
Tool Accuracy
97%
Reasoning Quality
94%
2. Select retrieval tool
Tool Accuracy
82%
Reasoning Quality
88%
3. Synthesize answer
Tool Accuracy
91%
Reasoning Quality
79%
4. Verify factuality
Tool Accuracy
76%
Reasoning Quality
71%
Error Categorization
18%41%29%12%
Wrong Tool18%
Hallucination41%
Plan Failure29%
Timeout12%
Controls
Task Type
Simulation runs50
Error Filter
Agent Evaluation measures trajectory efficiency (optimal vs actual steps) and error categories. Hallucination is the top error in QA; plan failures dominate in code generation. Efficiency below 0.6 usually signals over-planning or tool misuse.

Agent Evaluation - Trajectory and Error Analysis - Interactive Visualization

Evaluating LLM agents requires looking beyond final task success. Trajectory efficiency measures how many steps the agent used versus the optimal minimum. Step-by-step scores reveal where reasoning degrades. Error categories - wrong tool selection, hallucination, plan failure, and timeout - each require different mitigations. Research tasks show the lowest completion rates and highest hallucination rates.

  • Trajectory efficiency is optimal steps divided by actual steps - below 0.6 signals over-planning or tool misuse
  • Tool selection accuracy drops significantly at complex steps requiring multi-tool coordination
  • Hallucination is the dominant error in QA tasks; plan failure dominates in code generation
  • Step-by-step scoring pinpoints exactly which agent action type needs the most improvement

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.