SpatialEdit: Benchmarking Fine-Grained Image Spatial Editing
| Authors | Yicheng Xiao et al. |
| Year | 2026 |
| HF Upvotes | 35 |
| arXiv | 2604.04911 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Image spatial editing performs geometry-driven transformations, allowing precise control over object layout and camera viewpoints. Current models are insufficient for fine-grained spatial manipulations, motivating a dedicated assessment suite. Our contributions are listed: (i) We introduce SpatialEdit-Bench, a complete benchmark that evaluates spatial editing by jointly measuring perceptual plausibility and geometric fidelity via viewpoint reconstruction and framing analysis. (ii) To address the data bottleneck for scalable training, we construct SpatialEdit-500k, a synthetic dataset generated with a controllable Blender pipeline that renders objects across diverse backgrounds and systematic camera trajectories, providing precise ground-truth transformations for both object- and camera-centric operations. (iii) Building on this data, we develop SpatialEdit-16B, a baseline model for fine-grained spatial editing. Our method achieves competitive performance on general editing while substantially outperforming prior methods on spatial manipulation tasks. All resources will be made public at https://github.com/EasonXiao-888/SpatialEdit.
Engineering Breakdown
Plain English
This paper introduces SpatialEdit-Bench, a benchmark for evaluating image spatial editing models that transform object layouts and camera viewpoints with geometric precision. The authors also created SpatialEdit-500k, a synthetic dataset of 500,000 images generated in Blender with controlled camera trajectories and object placements, providing ground-truth geometric transformations for training and evaluation. The key innovation is jointly measuring both perceptual plausibility (does the edited image look realistic?) and geometric fidelity (is the spatial transformation mathematically correct?) through viewpoint reconstruction and framing analysis. This addresses a major gap: existing models struggle with fine-grained spatial manipulations, and there was no standardized way to assess whether edits preserve both visual quality and geometric accuracy.
Core Technical Contribution
The core novelty is a dual-metric evaluation framework that decouples visual plausibility from geometric correctness—two properties that can conflict in spatial editing tasks. Rather than relying on single-metric benchmarks (LPIPS, FID), the authors propose viewpoint reconstruction accuracy and framing analysis to explicitly measure whether spatial transformations match ground truth. The SpatialEdit-500k dataset is synthetically generated with precise control over camera parameters, object positions, and backgrounds, enabling scalable training without manual annotation bottlenecks. This addresses the critical limitation that prior spatial editing work had no large-scale, precisely-annotated dataset with verifiable ground-truth transformations.
How It Works
The pipeline works in two phases: benchmark design and dataset construction. For evaluation, SpatialEdit-Bench takes an input image and a spatial editing request (e.g., 'move camera left by 30 degrees' or 'shift object to right'), applies the edited image to a perceptual quality scorer and a geometric fidelity checker. The geometric fidelity component reconstructs the inferred camera viewpoint from the edited image and compares it against ground truth using reconstruction error; framing analysis measures whether object bounding boxes match expected positions after transformation. For data generation, the Blender pipeline renders 3D scenes with objects, cameras, and backgrounds, systematically varying camera trajectories (pan, tilt, zoom) and object placements while logging exact transformation parameters as ground truth. This allows models trained on SpatialEdit-500k to learn spatial transformations with precise supervision rather than indirect signals.
Production Impact
Engineers building spatial editing systems gain two immediate benefits: a rigorous benchmark to validate whether their models preserve both visual quality and geometric correctness, and a large-scale synthetic training dataset (500k images) that eliminates manual annotation effort. In production pipelines, adopting this benchmark means you can now catch failure modes where edits look visually plausible but geometrically wrong—a subtle but critical distinction for AR/VR applications, autonomous vehicle simulation, and 3D scene manipulation tools. The trade-off is that models must be trained on synthetic data, which introduces a domain gap between Blender renders and real photographs; practitioners will need fine-tuning on real data or domain adaptation techniques to deploy in production. The computational cost of generating 500k labeled images is amortized across all users of the benchmark, making it economically viable compared to manual annotation at scale.
Limitations and When Not to Use This
The paper does not address domain transfer from synthetic Blender renders to real-world photographs—a model trained purely on SpatialEdit-500k will likely struggle on user-submitted images due to lighting, texture, and reflectance differences. The benchmark assumes that geometric fidelity can be perfectly measured through viewpoint reconstruction, but this assumes the edited image preserves sufficient visual information to recover camera parameters; highly abstracted or stylized edits may fool the reconstruction metric. The paper does not evaluate complex multi-object scenes with occlusions or interactive object relationships (e.g., shadows cast between objects), limiting applicability to more intricate spatial edits. Additionally, the synthetic dataset may not capture rare real-world phenomena like specular highlights, motion blur, or extreme lighting conditions that violate the assumptions of the Blender pipeline.
Research Context
This work builds on the growing interest in geometry-aware image generation and editing, responding to limitations of diffusion-based image editing models (like InstructPix2Pix) that optimize for visual quality but ignore geometric constraints. It extends the benchmarking tradition established by datasets like COCO, Cityscapes, and ADE20k by introducing the first dedicated spatial editing benchmark with dual-metric evaluation. The paper is positioned as both a benchmark contribution (filling an evaluation gap) and a data contribution (providing 500k labeled examples), following the pattern of recent synthetic datasets like Objaverse and GSO that use procedural generation to solve data bottlenecks. This opens a research direction toward geometry-aware generative models that can jointly optimize perceptual quality and spatial correctness, relevant to emerging applications in 3D content creation, robotics simulation, and immersive media.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::