Generating Statistical Charts with Validation-Driven LLM Workflows
| Authors | Pavlin G. Poličar et al. |
| Year | 2026 |
| Field | Machine Learning |
| arXiv | 2605.00800 |
| Download | |
| Categories | cs.LG |
Abstract
Generating diverse, readable statistical charts from tabular data remains challenging for LLMs, as many failures become apparent after rendering and are not detectable from data or code alone. Existing chart datasets also rarely provide fully aligned artifacts, such as executable code, dataset context, and question-answer pairs. We present a structured LLM-based workflow that decomposes chart generation into dataset screening, plot proposal, code synthesis, rendering, validation-driven refinement, description generation, and question-answer generation. By incorporating rendered-output validation, the workflow addresses visualization-specific failure modes such as readability and semantic mismatch. It treats chart generation as an inspectable process rather than a one-shot prompt-to-code task, retaining each chart with its code, dataset context, description, and question-answer pairs. Applied to UCI datasets, the workflow produces 1,500 charts from 74 datasets, spanning 24 chart families and paired with 30,003 question-answer pairs. We evaluate 16 multimodal LLMs (MLLMs) on these chart-question pairs. The results show that chart-syntax questions are nearly saturated, while value extraction, comparison, and reasoning remain more challenging, illustrating the workflow's utility for diagnostic studies of chart-grounded multimodal reasoning.
Engineering Breakdown
Plain English
This paper presents a structured workflow for generating statistical charts from tabular data using LLMs, addressing a critical gap where many chart generation failures only become visible after the code is rendered rather than at the prompt or code level. The authors decompose chart generation into seven distinct stages: dataset screening, plot proposal, code synthesis, rendering, validation-driven refinement, description generation, and question-answer generation. The key innovation is incorporating rendered-output validation—actually running the code and inspecting the visual result—to catch visualization-specific failures like readability issues and semantic mismatches before finalizing the output. This transforms chart generation from a single-shot prompt-to-code task into an inspectable, iterative process with multiple feedback loops.
Core Technical Contribution
The core technical novelty is a validation-driven refinement loop that treats the rendered chart output as a first-class validation signal rather than a terminal output. Unlike prior one-shot code generation approaches, this workflow explicitly checks the visual properties of generated charts—axes labels, color contrast, data representation accuracy—and feeds that feedback back into the LLM for correction. The paper also addresses a dataset alignment problem: prior chart datasets lacked fully paired artifacts (executable code, dataset context, Q&A pairs), which they solve by structuring their workflow to produce all these artifacts together. This is a shift from treating chart generation as a code synthesis problem to treating it as a vision-aware iterative refinement problem.
How It Works
The workflow operates in a seven-stage pipeline starting with dataset screening, where the LLM evaluates whether the input tabular data is suitable for visualization and what chart types would be appropriate. Next, plot proposal generates candidate chart specifications (type, axes, grouping logic), followed by code synthesis which converts these specifications into executable Python or similar code. The rendering stage actually executes this code and captures the visual output as an image. Validation-driven refinement is the critical loop: the system analyzes the rendered image for failures (illegible labels, misaligned data, missing elements) and uses this visual feedback to prompt the LLM to revise the code. After a valid chart is produced, description generation creates natural language explanations of the chart, and finally question-answer generation creates paired examples for training or evaluation. The entire pipeline is orchestrated so that failures detected in rendering become explicit, actionable signals for code correction rather than silent errors.
Production Impact
For engineers building chart generation systems or data visualization pipelines, this approach eliminates a major class of hard-to-debug failures that occur in the rendering phase. Instead of deploying a model that generates code and hoping it produces readable charts, you now have an iterative validation loop that catches common visualization bugs: axis overflow, unreadable labels, color contrast issues, and data misalignment. The production trade-off is increased latency and compute—each chart generation now involves multiple LLM calls and code execution rounds rather than a single forward pass—but this is likely acceptable for batch chart generation, dashboard creation, or report generation systems where 2-5 seconds per chart is tolerable. Integration requires adding a rendering/visualization library (Matplotlib, Plotly) to your inference infrastructure and implementing the feedback loop logic, which adds complexity but is straightforward to build. The output consistency improvements would likely reduce downstream human review and correction cycles, offsetting the increased generation time for high-volume applications.
Limitations and When Not to Use This
The paper does not address scalability to very large datasets (millions of rows) or complex multi-layer visualizations that may exceed LLM context windows or rendering complexity. The approach assumes that visual inspection of rendered output is a reliable signal for correctness, but this may fail for domain-specific charts or statistical visualizations where semantic correctness depends on statistical assumptions rather than visual appearance. The workflow requires multiple forward passes through an LLM, making it substantially more expensive than single-shot approaches, which may be prohibitive for real-time or cost-sensitive applications. Additionally, the paper's completeness is unclear (abstract cuts off), but typically such work struggles with fully automatic detection of subtle failures like misleading axis ranges, inappropriate chart type selection, or incorrect statistical aggregations that don't manifest visually. The approach also depends on the underlying LLM's ability to understand and act on visual feedback, which may be limited for smaller or older models.
Research Context
This work extends the line of research on LLM-based code generation (exemplified by Codex and subsequent work) into the specialized domain of data visualization, where correctness requires both syntactic code validity and visual appropriateness. It builds on recent work in vision-language models and visual question-answering by treating rendered chart images as semantic signals that can guide generation, rather than only using code correctness metrics. The paper directly addresses gaps in existing chart generation datasets—prior work like PlotQA or ChartQA created large collections but without fully aligned executable code, natural language descriptions, and question-answer pairs, a misalignment that limits their utility for training end-to-end systems. This opens a research direction around multimodal feedback loops for code generation, where intermediate outputs (rendered visualizations, test results, visual properties) guide iterative refinement, potentially applicable to other domains like UI generation or CAD.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::