CurveBench: A Benchmark for Exact Topological Reasoning over Nested Jordan Curves
:::info Stub — Full Engineering Breakdown Coming This paper has a linked code implementation and was featured on Hugging Face Papers with 6 upvotes. A full breakdown with production viability rating, implementation notes, and honest limitations is being written. Subscribe to AI Letters → :::
| Authors | Amirreza Mohseni et al. |
| Year | 2026 |
| HF Upvotes | 6 |
| arXiv | 2605.14068 |
| Download | |
| Code | https://github.com/Amir-Mohseni/CurveBench |
Abstract
We introduce CurveBench, a benchmark for hierarchical topological reasoning from visual input. CurveBench consists of 756 images of pairwise non-intersecting Jordan curves across easy, polygonal, topographic-inspired, maze-like, and dense counting configurations. Each image is annotated with a rooted tree encoding the containment relations between planar regions. We formulate the task as structured prediction: given an image, a model must recover the full rooted containment tree induced by the curves. Despite the visual simplicity of the task, the strongest evaluated model, Gemini 3.1 Pro, achieves only 71.1% tree-generation accuracy on CurveBench-Easy and 19.1% on CurveBench-Hard. We further demonstrate benchmark utility through RLVR-style fine-tuning of open-weight vision-language models. Our trained Qwen3-VL-8B model improves over Qwen-3-VL-8B-Thinking from 2.8% to 33.3% tree-generation accuracy on CurveBench-Easy, exceeding GPT-5.4 and Claude Opus 4.5 under our evaluation protocol. The remaining gap, especially on CurveBench-Hard, shows that exact topology-aware visual reasoning remains far from solved.
Engineering Breakdown
Plain English
CurveBench is a new benchmark that tests whether vision-language models can perform exact topological reasoning—specifically, determining containment relationships between nested curves in images. The benchmark contains 756 images of non-intersecting Jordan curves with varying complexity, and the task is to output a tree structure encoding which regions are contained within which. Even the strongest model tested (Gemini 3.1 Pro) only achieves 71.1% accuracy on easy cases and 19.1% on hard cases, revealing a significant gap in spatial reasoning capabilities.
Key Engineering Insight
Topological reasoning—understanding spatial containment relationships—is a concrete, measurable capability gap in current vision-language models, and it requires structured output (tree generation) rather than free-form responses. This gap persists despite models having strong general capabilities, suggesting that hierarchical spatial reasoning is fundamentally harder than the models' overall performance metrics suggest.
Why It Matters for Engineers
Production systems relying on VLMs for spatial understanding (autonomous systems, document analysis, map interpretation, CAD/design tools) may silently fail on topological queries. This benchmark provides a precise way to measure and evaluate this failure mode before deploying models in safety-critical applications, rather than discovering the weakness in production.
Research Context
Prior VLM benchmarks focused on general visual understanding and object recognition, but didn't test precise geometric reasoning. CurveBench fills that gap by isolating a fundamental capability—spatial containment relationships—that's essential for many real-world tasks. The benchmark's structured prediction framing (output trees, not descriptions) also pushes toward measurable, verifiable spatial reasoning rather than fuzzy language-based answers.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::