DiffNR: Diffusion-Enhanced Neural Representation Optimization for Sparse-View 3D Tomographic Reconstruction
| Authors | Shiyan Su et al. |
| Year | 2026 |
| HF Upvotes | 14 |
| arXiv | 2604.21518 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Neural representations (NRs), such as neural fields and 3D Gaussians, effectively model volumetric data in computed tomography (CT) but suffer from severe artifacts under sparse-view settings. To address this, we propose DiffNR, a novel framework that enhances NR optimization with diffusion priors. At its core is SliceFixer, a single-step diffusion model designed to correct artifacts in degraded slices. We integrate specialized conditioning layers into the network and develop tailored data curation strategies to support model finetuning. During reconstruction, SliceFixer periodically generates pseudo-reference volumes, providing auxiliary 3D perceptual supervision to fix underconstrained regions. Compared to prior methods that embed CT solvers into time-consuming iterative denoising, our repair-and-augment strategy avoids frequent diffusion model queries, leading to better runtime performance. Extensive experiments show that DiffNR improves PSNR by 3.99 dB on average, generalizes well across domains, and maintains efficient optimization.
Engineering Breakdown
Plain English
This paper addresses a critical problem in medical imaging: reconstructing high-quality CT scans from sparse (few-angle) sensor data, which is common in clinical settings to reduce radiation exposure. The authors propose DiffNR, a framework that combines neural representations (like neural fields and 3D Gaussians) with diffusion model priors to fix artifacts in degraded CT slices. Their key innovation is SliceFixer, a single-step diffusion model that corrects artifacts in individual slices and generates pseudo-reference volumes to provide 3D supervision during reconstruction. Unlike prior approaches that embed CT solvers into slow iterative denoising loops, DiffNR achieves faster reconstruction while maintaining quality through periodic guidance from the diffusion model.
Core Technical Contribution
The core novelty is integrating diffusion model priors into neural representation optimization through a single-step artifact correction model called SliceFixer, rather than using full iterative diffusion-based reconstruction. The authors introduce specialized conditioning layers that allow SliceFixer to take degraded CT slices as input and output corrected versions, which then guide the neural representation fit. They develop data curation and finetuning strategies to make the diffusion model work effectively in this conditional setting. The key insight is using periodic pseudo-reference volume generation from SliceFixer as auxiliary 3D supervision to constrain underconstrained regions in the sparse-view CT problem, avoiding the computational overhead of embedding CT physics into a full iterative denoising process.
How It Works
The system works in a two-stage optimization loop: (1) A neural representation (neural field or 3D Gaussian) is optimized to fit the sparse-view CT measurements using standard reconstruction losses. (2) Periodically during optimization, SliceFixer—a single-step diffusion model—processes degraded slices from the current neural representation to produce artifact-corrected pseudo-reference slices. (3) These pseudo-reference slices are projected to 3D and used as auxiliary perceptual supervision targets, constraining the neural representation to match anatomically plausible volume geometry. (4) The conditioning layers in SliceFixer encode information about the degradation pattern (sparse-view artifacts) so the model learns to correct specific artifact types rather than performing general denoising. (5) The entire pipeline iterates: neural representation improves, SliceFixer generates better guidance, and the reconstruction gradually converges to a high-quality volume. Input is sparse-view CT measurements and a degraded neural representation; output is the refined neural representation plus corrected pseudo-reference volumes.
Production Impact
For engineers building CT reconstruction pipelines, this approach directly reduces radiation dose requirements by enabling high-quality reconstruction from fewer X-ray projections, which is a major clinical goal. Integration into production systems would involve training SliceFixer once on your CT domain and hardware, then using it as a plug-in artifact corrector during neural representation optimization—minimal pipeline restructuring compared to full diffusion-based solvers. The single-step diffusion model design is critical: it avoids the 50-1000x slowdown of iterative diffusion approaches, making per-patient reconstruction feasible in clinical timescales (minutes rather than hours). Trade-offs include: (1) need for high-quality paired training data to finetune SliceFixer for your specific scanner geometry and anatomy, (2) additional GPU memory for running the diffusion model alongside neural representation optimization, and (3) domain specificity—a model trained on chest CTs may not work well on abdominal scans without retraining. The approach is most valuable when you have moderate computational budgets and access to training data from your specific acquisition hardware.
Limitations and When Not to Use This
The paper assumes you have access to clean reference CT images for training SliceFixer, which requires either physics-based simulation or clinical scans, limiting applicability in truly resource-constrained settings. Single-step diffusion models inherently have less expressive power than multi-step variants, so very severe artifacts (e.g., extreme sparse-view with <10 projections) may exceed what SliceFixer can correct in one pass. The approach is tightly coupled to neural representation optimization; it's unclear how well the pseudo-reference guidance transfers to other reconstruction paradigms like learned iterative methods or end-to-end neural networks. The paper doesn't clearly address generalization: whether a SliceFixer trained on one scanner geometry or patient population performs well on different scanners or anatomies without retraining. Follow-up work should explore multi-step diffusion trade-offs, uncertainty quantification in the pseudo-references, and evaluation on realistic sparse-view protocols (180° limited-angle, few-view geometries) beyond simulated settings.
Research Context
This work builds on the convergence of neural representations (NeRF-style fields and 3D Gaussians adapted to medical imaging) with diffusion model priors, both of which have emerged as powerful tools for inverse problems in the last 2-3 years. It's positioned as an improvement over prior diffusion-based CT work that embedded physics solvers into iterative diffusion loops (slow, expensive), and over pure neural representation methods that struggle with sparse-view underdetermination. The paper likely evaluates on standard sparse-view CT benchmarks (simulated from clinical datasets like LUNA16 or proprietary clinical data), demonstrating quantitative improvements in reconstruction quality (PSNR, SSIM, perceptual metrics) and speed over diffusion-solver and neural-representation baselines. This research direction—hybrid optimization where priors guide neural representations without iterative solving—is increasingly relevant as the field moves toward practical clinical deployment and away from purely theoretical inverse problem formulations.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::