Stop Holding Your Breath: CT-Informed Gaussian Splatting for Dynamic Bronchoscopy
| Authors | Andrea Dunn Beltran et al. |
| Year | 2026 |
| Field | Computer Vision |
| arXiv | 2604.28179 |
| Download | |
| Categories | cs.CV |
Abstract
Bronchoscopic navigation relies on registering endoscopic video to a preoperative CT scan, but respiratory motion deforms the airway by 5-20 mm, creating CT-to-body divergence that limits localization accuracy. In practice, this is mitigated through breath-hold protocols, which attempt to match the intraoperative anatomy to a static CT, but are difficult to reproduce and disrupt clinical workflow. We propose to eliminate the need for breath-hold protocols by leveraging patient-specific respiratory modeling. Paired inhale-exhale CT scans, already acquired for planning, implicitly define the patient-specific deformation space of the breathing airway. By registering these scans, we reduce respiratory motion to a single scalar breathing phase per frame, constraining all reconstructions to anatomically observed configurations. We embed this representation within a mesh-anchored Gaussian splatting framework, where a lightweight estimator infers breathing phase directly from endoscopic RGB, enabling continuous, deformation-aware reconstruction throughout the respiratory cycle without breath-holds or external sensing. To enable quantitative evaluation, we introduce RESPIRE, a physically grounded bronchoscopy simulation pipeline with per-frame ground truth for geometry, pose, breathing phase, and deformation. Experiments on RESPIRE show that our approach achieves geometrically faithful reconstruction, over 20x faster training, and 1.22 mm target localization accuracy (within the 3mm clinically relevant tolerances) outperforming unconstrained single-CT baselines. Please check out our website for additional visuals: https://asdunnbe.github.io/RESPIRE/
Engineering Breakdown
Plain English
This paper tackles respiratory motion in bronchoscopic surgery by using paired inhale-exhale CT scans to build a patient-specific breathing model that eliminates the need for difficult breath-hold protocols during surgery. The core problem is that breathing deforms airways by 5-20mm, causing misalignment between the preoperative CT scan and the actual patient anatomy during the procedure, which degrades surgical navigation accuracy. The authors' key insight is that by registering the inhale and exhale scans, they can reduce the entire breathing deformation to a single scalar value per video frame, constraining all reconstructions to anatomically valid states. This approach promises to improve clinical workflow and localization accuracy without requiring surgeons to ask patients to hold their breath at specific moments.
Core Technical Contribution
The core novelty is framing respiratory motion as a low-dimensional, patient-specific deformation space that can be captured by registering paired inhale-exhale CT scans acquired during standard preoperative planning. Rather than using generic breathing models or asking for breath-hold compliance, the authors exploit the fact that these paired scans implicitly define the full range of realistic anatomical deformations for that specific patient's airway. By reducing the infinite complexity of breathing motion to a single scalar breathing phase parameter per frame, they create a highly constrained reconstruction problem that keeps all intraoperative estimates within the patient's actual anatomical range. This is a shift from traditional registration approaches that either ignore respiratory motion or require explicit patient cooperation during surgery.
How It Works
The system takes as input paired preoperative CT scans (inhale and exhale states) and real-time endoscopic video frames during the procedure. First, the authors register these two CT scans to establish a deformation field that maps between breathing states, effectively creating a continuous model of how the airway deforms as the patient breathes. For each video frame captured during surgery, the system estimates a single breathing phase parameter (a scalar between 0 and 1, representing position along the inhale-exhale spectrum) rather than trying to directly register the video to one static CT. Using this breathing phase, the system interpolates the appropriate deformed CT state and registers the video endoscopy to that deformed anatomy, rather than to a fixed breath-hold state. The key architectural advantage is that this constrains the search space dramatically—instead of registering to arbitrary 3D anatomy, the algorithm only searches for the best breathing phase, reducing a high-dimensional optimization problem to a 1D search. This makes the system more robust and faster than traditional CT-to-video registration methods.
Production Impact
For surgical navigation systems, this eliminates the clinical friction of breath-hold protocols, which are unreliable, disrupt workflow, and require patient training and repeated attempts. Surgeons would see improved localization accuracy during live bronchoscopy because the system adapts to the patient's actual breathing rather than assuming a static anatomy; this directly translates to safer, more precise interventions and shorter procedure times. The implementation cost is minimal since the paired CT scans are already acquired for surgical planning—no new imaging hardware or sequences are required. However, the system does add a new dependency: the registration between inhale and exhale CTs must be accurate and robust, and the breathing phase estimation from endoscopic video must be reliable in challenging lighting and view conditions. The latency impact is favorable since you're doing 1D optimization instead of full 3D registration per frame, but you need to ensure the deformation model generalizes to the actual intraoperative breathing patterns (which may differ from quiet breathing during the planning CT).
Limitations and When Not to Use This
The approach assumes that the patient's intraoperative breathing remains within the envelope captured by the inhale-exhale planning scans, which may not hold during stress, anesthesia, or if the patient's respiratory pattern changes during the procedure. The method depends critically on accurate registration of the inhale and exhale CT scans, and errors in this foundational step propagate through all downstream reconstructions; the paper does not clearly specify tolerance thresholds or robustness to registration failure. Breathing phase estimation from endoscopic video alone is underconstrained—the system must infer a 1D breathing phase from 2D video features, which could fail in pathological anatomies, severe deformation, or when distinctive landmarks are not visible. The approach is specific to bronchoscopy and may not generalize to other organs with different respiratory mechanics or to procedures where non-respiratory deformation dominates (e.g., patient movement, soft tissue compression).
Research Context
This work builds on a long line of research in image-guided surgery and deformable image registration, particularly methods that constrain registration to anatomically plausible states. It extends prior work in respiratory motion modeling—historically, the field has used either generic population-based breathing models or relied on external markers and breath-hold protocols, but this paper uniquely leverages patient-specific motion data already present in the clinical workflow. The contribution is relevant to the broader challenge of temporal alignment and motion compensation in endoscopic video, an active area with applications beyond bronchoscopy (e.g., colonoscopy, laparoscopy). This opens a research direction toward learning-based breathing phase estimation from endoscopic video features and toward multi-organ deformation models that could extend the approach to other procedures with respiratory motion.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::