Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Data

Authors	Saptarshi Chakraborty et al.
Year	2026
Field	Statistics / ML
arXiv	2603.03700
PDF	Download
Categories	stat.ML, cs.AI, cs.LG

Abstract

Despite the remarkable empirical success of score-based diffusion models, their statistical guarantees remain underdeveloped. Existing analyses often provide pessimistic convergence rates that do not reflect the intrinsic low-dimensional structure common in real data, such as that arising in natural images. In this work, we study the statistical convergence of score-based diffusion models for learning an unknown distribution $μ$ from finitely many samples. Under mild regularity conditions on the forward diffusion process and the data distribution, we derive finite-sample error bounds on the learned generative distribution, measured in the Wasserstein-$p$ distance. Unlike prior results, our guarantees hold for all $p \ge 1$ and require only a finite-moment assumption on $μ$, without compact-support, manifold, or smooth-density conditions. Specifically, given $n$ i.i.d.\ samples from $μ$ with finite $q$-th moment and appropriately chosen network architectures, hyperparameters, and discretization schemes, we show that the expected Wasserstein-$p$ error between the learned distribution $\hatμ$ and $μ$ scales as \mathbb{E}\, \mathbb{W}_p(\hatμ,μ) = \widetilde{O}\!\left(n^{-1 / d^\ast_{p,q}(μ)}\right), where d^\ast_{p,q}(μ) is the $(p,q)$-Wasserstein dimension of $μ$. Our results demonstrate that diffusion models naturally adapt to the intrinsic geometry of data and mitigate the curse of dimensionality, since the convergence rate depends on d^\ast_{p,q}(μ) rather than the ambient dimension. Moreover, our theory conceptually bridges the analysis of diffusion models with that of GANs and the sharp minimax rates established in optimal transport. The proposed $(p,q)$-Wasserstein dimension also extends classical Wasserstein dimension notions to distributions with unbounded support, which may be of independent theoretical interest.

---

Engineering Breakdown

Plain English

This paper develops the first rigorous statistical convergence guarantees for score-based diffusion models that actually match empirical performance on real datasets. Prior theoretical analyses provided overly pessimistic bounds that didn't account for the low-dimensional structure present in natural images and other real-world data. The authors derive finite-sample error bounds measured in Wasserstein-p distance that hold for all p ≥ 1 under only mild regularity conditions on the forward diffusion process and a finite-moment assumption on the data distribution. The key finding is that their analysis captures how diffusion models exploit intrinsic data geometry, unlike previous work that treated data as if it were uniformly distributed in high-dimensional space.

Core Technical Contribution

The core novelty is a statistical analysis framework that directly incorporates low-dimensional structure into convergence rate guarantees for diffusion models. Instead of analyzing diffusion models as operating in ambient high-dimensional space (which yields exponential dependence on dimension), the authors prove convergence rates that depend only on intrinsic dimension properties of the target distribution μ. Their bounds work for the entire Wasserstein-p family of distances (p ≥ 1) rather than just specific distances, and they require only finite-moment assumptions rather than compact support, making assumptions more aligned with real distributions. This is technically achieved through refined analysis of the score-matching loss and how it propagates through the forward and reverse diffusion processes.

How It Works

The analysis starts with a score-based diffusion model that learns to estimate ∇log p_t(x) at each timestep t of the forward process. The forward process gradually adds Gaussian noise to samples from the unknown distribution μ according to a predetermined schedule. During training, the model estimates scores at all noise levels using empirical samples, and convergence is measured by how closely the learned score function approximates the true score. The reverse process then starts from pure noise and iteratively denoise using the learned scores to generate new samples. The key technical insight is that the authors decompose the total error into: (1) approximation error from the parameterized score network, (2) statistical estimation error from finite samples, and (3) discretization error from using finite timesteps. They show that when the data lies on a low-dimensional manifold or has low intrinsic dimension, the approximation requirements scale with that intrinsic dimension rather than ambient dimension, directly improving convergence rates.

Production Impact

For engineers deploying generative models, this work provides theoretical justification for why diffusion models work well on natural images despite theoretical fears about curse of dimensionality. It validates architectural choices and training procedures by proving they achieve good generalization with reasonable sample complexity. In practice, this means you can have confidence in scaling diffusion models to production without needing massive datasets relative to ambient dimension—the guarantees show that intrinsic structure is automatically exploited. However, the analysis still assumes access to accurate score estimation and well-behaved forward processes; in production you'd need to validate these assumptions apply to your domain. The main trade-off is that while guarantees now exist, they're still looser than empirical performance (theory lags practice), so this is validation rather than a fundamentally new capability.

Limitations and When Not to Use This

The analysis requires regularity conditions on the forward diffusion process and assumes the data distribution μ satisfies mild smoothness or moment conditions that may not hold for all real datasets (e.g., multimodal distributions with heavy tails). The paper doesn't address practical concerns like score network parameterization choices, numerical stability of diffusion ODEs, or how to estimate intrinsic dimension in practice—it assumes ideal estimation. The bounds, while dimension-dependent, are still not tight enough to precisely predict sample complexity for specific problems, so practitioners can't directly use these results to size datasets. Additionally, the analysis covers unconditional generation; extension to conditional generation (text-to-image, etc.) and other variants like guidance mechanisms is left to future work.

Research Context

This work builds directly on the score-based generative modeling framework established by Song et al. and recent advances in diffusion model theory by researchers like Hyvarinen. It addresses a major gap: while empirical success of diffusion models (OpenAI DALL-E, Stable Diffusion) is undisputed, theoretical understanding was lagging far behind, with existing analyses suggesting exponential sample complexity that contradicts practice. The paper fits into a broader research direction on closing the theory-practice gap in deep generative models, following similar efforts for GANs and VAEs. This opens new directions for analyzing other generative models through the lens of intrinsic geometry and for developing better initialization/architecture strategies that provably exploit low-dimensional structure.

---

:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::

---

Back to Research Lab → · Subscribe to AI Letters →