LLM Observability - Trace Waterfall - Interactive Visualization

Observability for LLM applications means tracing every request through each processing stage: tokenization, optional RAG retrieval, generation, output filtering, and response delivery. A distributed trace waterfall shows each span as a horizontal bar, with width proportional to duration. P50/P95/P99 latency percentiles reveal tail latency issues - a slow P99 means 1 in 100 users has a terrible experience even if the median is fine. Error traces show retry patterns and the cost of failures.

• Span waterfall: each processing stage rendered as a horizontal bar with exact ms duration
• Four trace types: fast (188ms), slow (4.3s), RAG-augmented (758ms), error with retries
• P50/P95/P99 latency cards for each trace type
• Token count display at each stage - see where tokens are consumed
• Error indicator: red spans highlight failed attempts, shows retry overhead
• Used in: LangSmith, Langfuse, Arize, Helicone, OpenTelemetry + custom dashboards

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.