CylinderDepth: Cylindrical Spatial Attention for Multi-View Consistent Self-Supervised Surround Depth Estimation
| Authors | Samer Abualhanud et al. |
| Year | 2026 |
| HF Upvotes | 2 |
| arXiv | 2511.16428 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Self-supervised surround-view depth estimation enables dense, low-cost 3D perception with a 360° field of view from multiple minimally overlapping images. Yet, most existing methods suffer from depth estimates that are inconsistent across overlapping images. To address this limitation, we propose a novel geometry-guided method for calibrated, time-synchronized multi-camera rigs that predicts dense metric depth. Our approach targets two main sources of inconsistency: the limited receptive field in border regions of single-image depth estimation, and the difficulty of correspondence matching. We mitigate these two issues by extending the receptive field across views and restricting cross-view attention to a small neighborhood. To this end, we establish the neighborhood relationships between images by mapping the image-specific feature positions onto a shared cylinder. Based on the cylindrical positions, we apply an explicit spatial attention mechanism, with non-learned weighting, that aggregates features across images according to their distances on the cylinder. The modulated features are then decoded into a depth map for each view. Evaluated on the DDAD and nuScenes datasets, our method improves both cross-view depth consistency and overall depth accuracy compared with state-of-the-art approaches. Code is available at https://abualhanud.github.io/CylinderDepthPage.
Engineering Breakdown
Plain English
This paper addresses a key limitation in self-supervised 3D perception for autonomous vehicles and robotics: existing methods that estimate depth from multiple camera views produce inconsistent depth maps at image boundaries and where views overlap. The authors propose a geometry-guided deep learning approach that leverages synchronized multi-camera rigs (like surround-view camera systems on cars) to predict dense, metric-calibrated depth across 360° with improved consistency. Their method works by extending the receptive field of the depth estimator across neighboring camera views and using localized cross-view attention to handle the correspondence matching problem, which is traditionally a hard constraint in 3D reconstruction. The result is depth estimates that remain consistent where images overlap, reducing artifacts that currently plague production depth estimation systems.
Core Technical Contribution
The core novelty is a geometry-aware cross-view depth aggregation mechanism specifically designed for multi-camera rigs with known calibration and synchronization. Rather than estimating depth independently per image (the standard self-supervised approach), the authors explicitly model camera geometry and use it to constrain attention across views, significantly reducing the search space for matching correspondences. They identify and directly target two specific failure modes: (1) border regions in single-image depth networks have limited receptive fields and thus poor depth estimates, and (2) corresponding pixels across different views are difficult to match due to the large search space. By restricting cross-view attention to a small neighborhood derived from camera geometry, they make the correspondence problem tractable while leveraging geometric priors that existing self-supervised methods ignore.
How It Works
The input is synchronized, calibrated multi-camera image streams from a rig (e.g., 4–6 surround-view cameras on a vehicle). The method first extracts per-image features using a backbone encoder on each camera view independently. For each pixel, the method then projects its 3D neighborhood (a small epipolar band) from the current view into neighboring camera views using the known camera calibration matrix. This geometric projection creates a small search region—the 'neighborhood'—for matching candidates. Cross-view attention is restricted to pixels within this neighborhood, replacing the expensive all-to-all attention that would normally be needed for correspondence matching. The attention mechanism learns to select the correct matching pixel within this constrained region, then uses the matched features to refine depth estimates. Finally, a depth decoder aggregates the cross-view information and outputs dense metric depth (in absolute units like meters) that is consistent across views. The geometric constraint acts as a strong inductive bias that guides learning and ensures the model respects camera projection geometry.
Production Impact
For engineers building autonomous vehicle or robotics perception stacks, this solves a critical problem: surround-view depth systems today produce discontinuities and artifacts at image boundaries and in overlap regions, which break downstream planning and collision avoidance logic. Adopting this approach would allow you to train depth estimators end-to-end without manual depth labels (self-supervised), yet get metric-calibrated, view-consistent outputs—eliminating the need for expensive LiDAR supervision or multi-stage post-processing heuristics. The geometric priors mean the model is more sample-efficient and generalizes better to new environments than purely geometric-agnostic approaches. Computationally, the method requires multi-view processing and cross-attention at inference, so latency depends on the number of cameras and the size of the neighborhood; for typical 6-camera surround rigs, this is likely 1.5–3× the cost of single-image estimation, but still well within real-time budgets (< 100ms on modern GPUs). The main trade-off is that you need well-calibrated, time-synchronized cameras—a requirement that already exists in production rigs, so this is not a new burden.
Limitations and When Not to Use This
The method assumes cameras are properly calibrated and time-synchronized, assumptions that break down in real-world deployments where cameras drift or clocks skew; the paper does not explore robustness to calibration errors or temporal misalignment, which are common in production. The approach is designed for static or slowly moving scenes and may struggle with fast motion or dynamic objects that violate the geometric constraints baked into the neighborhood definition. Because it relies on self-supervised learning (photometric consistency losses), it will fail in textureless regions and reflective surfaces (glass, water) where pixel matching is inherently ambiguous—a fundamental limitation of photometric loss that the paper inherits. The paper also does not address how to handle view occlusions, camera failures, or night driving scenarios with poor visibility, all common in production. Finally, the bounded receptive field across views may still miss long-range context needed for semantic understanding of depth discontinuities (e.g., thin poles or far-away objects), so this method is best paired with explicit object detection rather than used in isolation.
Research Context
This work builds on a decade of self-supervised monocular depth estimation research (Monodepth, PackNet, MonoDepth2) which showed that depth can be learned without labels by enforcing photometric consistency between views. However, those methods were single-image approaches and produced inconsistent depth at boundaries and in multi-camera settings. More recent work on multi-view stereo and neural radiance fields (NeRF) showed that geometric constraints can improve 3D reconstruction, but those methods are expensive at inference and require dense view overlap. This paper's novelty is applying geometric priors specifically to the multi-camera rig setting—a practical constraint that most academic papers ignore but that dominates real robotics and autonomous driving. It opens a research direction toward learned 3D perception that bakes in camera geometry from the outset, rather than trying to learn geometry implicitly. Future work could extend this to handle dynamic objects, occlusions, and camera failures, or combine it with semantic segmentation to improve depth in ambiguous regions.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::