DV-World: Benchmarking Data Visualization Agents in Real-World Scenarios
| Authors | Jinxiang Meng et al. |
| Year | 2026 |
| HF Upvotes | 40 |
| arXiv | 2604.25914 |
| Download | |
| Code | https://github.com/DA-Open/DV-World |
Abstract
Real-world data visualization (DV) requires native environmental grounding, cross-platform evolution, and proactive intent alignment. Yet, existing benchmarks often suffer from code-sandbox confinement, single-language creation-only tasks, and assumption of perfect intent. To bridge these gaps, we introduce DV-World, a benchmark of 260 tasks designed to evaluate DV agents across real-world professional lifecycles. DV-World spans three domains: DV-Sheet for native spreadsheet manipulation including chart and dashboard creation as well as diagnostic repair; DV-Evolution for adapting and restructuring reference visual artifacts to fit new data across diverse programming paradigms and DV-Interact for proactive intent alignment with a user simulator that mimics real-world ambiguous requirements. Our hybrid evaluation framework integrates Table-value Alignment for numerical precision and MLLM-as-a-Judge with rubrics for semantic-visual assessment. Experiments reveal that state-of-the-art models achieve less than 50% overall performance, exposing critical deficits in handling the complex challenges of real-world data visualization. DV-World provides a realistic testbed to steer development toward the versatile expertise required in enterprise workflows. Our data and code are available at https://github.com/DA-Open/DV-World{this project page}.
Engineering Breakdown
Plain English
DV-World is a benchmark with 260 real-world data visualization tasks designed to evaluate AI agents on their ability to create, repair, and adapt charts and dashboards across spreadsheets, multiple programming languages, and interactive user scenarios. Unlike existing benchmarks that run code in isolated sandboxes and assume users know exactly what they want, DV-World tests agents in three realistic domains: native spreadsheet operations (DV-Sheet), adapting visualizations to new data across different platforms (DV-Evolution), and handling ambiguous user requirements through interaction (DV-Interact). The benchmark moves beyond single-language, create-only tasks to assess the full lifecycle of professional data visualization work. This addresses a critical gap in AI benchmarking by grounding agents in real environmental constraints and user communication challenges that production systems actually encounter.
Core Technical Contribution
The core novelty is a multi-domain benchmark framework that extends beyond the typical code-sandbox evaluation paradigm to include three previously under-explored aspects of real-world visualization work: (1) native platform manipulation within actual spreadsheet environments rather than isolated Python execution, (2) cross-platform evolution that requires adapting existing visualizations to new datasets using different programming paradigms, and (3) interactive intent alignment through user simulation that models the ambiguity and back-and-forth negotiation of real professional communication. Rather than treating visualization as a pure code generation problem, DV-World treats it as a situated interaction problem where agents must understand environmental constraints, versioning/adaptation requirements, and handle incomplete specifications. This shifts the evaluation paradigm from 'can the model generate correct code?' to 'can the agent operate effectively in real professional data workflows?'
How It Works
DV-World operates as a three-component evaluation framework. First, DV-Sheet tasks present agents with spreadsheet environments (Excel, Google Sheets) where they must create or repair charts and dashboards using native tools and formulas, requiring interaction with actual UI elements rather than code-only APIs. Second, DV-Evolution tasks provide reference visualizations alongside new datasets, and agents must translate and adapt the visual design across different programming languages (Python, R, JavaScript) and libraries, testing their understanding of design intent versus specific implementation. Third, DV-Interact tasks use a user simulator that presents ambiguous requirements in natural language, requires agents to ask clarifying questions, and iteratively refines specifications until the correct visualization is produced. The benchmark evaluates both task completion (whether the final visualization matches ground truth) and process metrics (how many interactions were needed, whether the agent correctly diagnosed ambiguities). Agents are evaluated across all 260 tasks with automatic verification through visual comparison, schema matching, and code equivalence checking.
Production Impact
For engineers building data visualization tools or AI-assisted analytics platforms, DV-World provides a realistic evaluation framework that goes beyond academic toy problems. If you adopt this benchmark, you immediately surface failure modes that code-sandbox testing misses: agents may generate syntactically correct code that breaks in actual spreadsheet environments, fail to adapt designs when data schemas change, or frustrate users by not asking clarifying questions when requirements are ambiguous. This directly impacts product quality—a production visualization agent needs to handle the complete lifecycle (create, maintain, evolve, clarify) not just initial generation. The interactive component forces you to build proper intent-understanding capabilities, which increases architectural complexity but directly maps to user satisfaction. Trade-offs include higher evaluation cost (interactive tasks require human-simulator overhead), longer benchmark runtime (native platform testing can't be parallelized as easily as code execution), and the need to support multiple platforms and languages rather than a single code generation target.
Limitations and When Not to Use This
DV-World does not address scalability to large-scale enterprise visualization systems with hundreds of datasets and complex governance requirements—the benchmark focuses on individual visualization tasks, not system-wide coordination. The user simulator, while valuable, is still a synthetic approximation of real human ambiguity and may not capture the full distribution of genuine user confusion or domain-specific context that actual professionals bring. The benchmark also assumes access to native environments (actual Excel/Google Sheets instances), which creates infrastructure complexity for researchers without enterprise tooling access and may bias evaluation toward certain platforms or tools. Additionally, the paper does not deeply explore failure recovery or graceful degradation when agents encounter unsupported features or platform limitations—it primarily tests whether agents can complete tasks, not how robustly they handle edge cases or communicate constraints back to users. The 260-task scale, while substantial, may not be sufficient to train or fine-tune large models without significant overfitting risk.
Research Context
DV-World builds on a growing body of work recognizing that AI benchmarks often oversimplify real-world requirements. It extends prior code-generation benchmarks (like HumanEval, DS-1000) by introducing environmental grounding and interactive components inspired by embodied AI and interactive machine learning research. The work aligns with recent shifts toward evaluating AI agents in real systems rather than isolated code execution environments (similar to efforts in robotics benchmarking and web automation). The interactive intent-alignment component draws from dialogue systems and clarification research, recognizing that perfect specifications are unrealistic. By spanning multiple domains (spreadsheets, multiple programming languages, user interaction), DV-World opens research directions in cross-platform abstraction, domain-specific intent understanding, and multi-turn agent planning. This positions visualization as a domain where agents must reason about multiple objectives simultaneously (correctness, maintainability, user intent) rather than a pure code generation task.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::