Generating DDPM-based Samples from Tilted Distributions
| Authors | Himadri Mandal et al. |
| Year | 2026 |
| Field | Machine Learning |
| arXiv | 2604.03015 |
| Download | |
| Categories | cs.LG, stat.ML |
Abstract
Given independent samples from a -dimensional probability distribution, our aim is to generate diffusion-based samples from a distribution obtained by tilting the original, where the degree of tilt is parametrized by θ\in \mathbb{R}^d. We define a plug-in estimator and show that it is minimax-optimal. We develop Wasserstein bounds between the distribution of the plug-in estimator and the true distribution as a function of and , illustrating regimes where the output and the desired true distribution are close. Further, under some assumptions, we prove the TV-accuracy of running Diffusion on these tilted samples. Our theoretical results are supported by extensive simulations. Applications of our work include finance, weather and climate modelling, and many other domains, where the aim may be to generate samples from a tilted distribution that satisfies practically motivated moment constraints.
Engineering Breakdown
Plain English
This paper solves the problem of generating samples from tilted (modified) versions of a learned probability distribution using diffusion models. Given n samples from a d-dimensional distribution, the authors develop a method to create new samples from an exponentially-tilted version of that distribution, where the tilt strength is controlled by a parameter θ. They prove their plug-in estimator is minimax-optimal and provide Wasserstein bounds showing when generated samples match the true tilted distribution. The work includes theoretical guarantees on TV-accuracy and is validated with simulations on finance, weather, and climate modeling tasks.
Core Technical Contribution
The core novelty is formalizing how to generate samples from tilted distributions using diffusion models with theoretical optimality guarantees. Unlike prior work that estimates distributions then samples from them separately, this paper directly characterizes the estimation-to-sampling pipeline with minimax-optimal rates and derives explicit Wasserstein bounds that depend on both sample size n and tilt parameter θ. The key insight is proving that running diffusion on tilted empirical samples produces TV-accurate outputs under stated assumptions, bridging the gap between tilted distribution theory and practical diffusion model deployment. This is the first work to provide end-to-end theoretical guarantees for DDPM-based sampling from tilted distributions.
How It Works
The method takes n independent samples from an original d-dimensional distribution and constructs a plug-in estimator of the exponentially-tilted distribution p(x) ∝ exp(θ·x)p₀(x), where p₀ is the original distribution and θ controls the tilt direction and strength. The estimator tilts the empirical distribution by reweighting samples according to the tilt parameter θ, creating a finite sample approximation of the true tilted distribution. This tilted sample set is then fed directly into a standard DDPM (Denoising Diffusion Probabilistic Model) to generate new samples. The paper proves two things: (1) the Wasserstein distance between the plug-in estimator's distribution and the true tilted distribution scales appropriately with n and θ, and (2) running diffusion on these tilted samples produces outputs that match the true tilted distribution in total variation distance under stated assumptions. The theoretical analysis tracks how estimation error propagates through the diffusion sampling process.
Production Impact
For engineers building financial or climate modeling systems, this enables shifting a learned model's behavior toward high-impact regions without retraining. Instead of collecting new labeled data or fine-tuning models, you can generate samples biased toward desired outcomes (e.g., high-return portfolios, extreme weather scenarios) by specifying a tilt vector θ and running diffusion. This reduces data collection and compute costs compared to retraining while maintaining theoretical guarantees on sample quality. The main trade-off is computational: you need a pre-trained diffusion model and θ must be tuned for your application. Integration is straightforward—apply the tilt during diffusion inference, not in the training pipeline—making this a low-friction addition to existing DDPM deployments. Latency impact is minimal since diffusion cost is dominated by the iterative denoising process, not the tilting step.
Limitations and When Not to Use This
The method requires that the original distribution and tilted distribution are well-behaved enough for diffusion models to sample from effectively; it does NOT guarantee good results if the tilt θ is so extreme that the tilted distribution becomes multimodal or pathological. The paper assumes you already have a trained diffusion model and samples from the original distribution—it does NOT address how to train the diffusion model or handle the case where you only have samples from a different distribution. The bounds depend on dimensionality d and sample size n, so scalability in very high dimensions (>1000) is not empirically validated. The work also assumes the tilt is applied to an exponential family form θ·x; more complex tilt functions or conditional tilting are not covered. Practical deployment requires careful tuning of θ and validation that TV-accuracy bounds actually hold for your specific data distribution.
Research Context
This work extends classical results in exponential tilting and importance sampling from the parametric setting to the modern diffusion model regime. It builds on recent theoretical advances in diffusion model convergence (e.g., L2-accuracy guarantees for DDPM) and connects them to distributional tilting, a well-studied problem in statistics with applications in rare-event simulation and likelihood-free inference. The paper advances the broader trend of adding theoretical guarantees to generative models, following work on score-based models and likelihood bounds. It opens up new applications in likelihood-free simulation for complex systems (weather, finance) where you can specify desired outcomes via θ rather than collecting more labeled data, positioning diffusion models as practical tools for importance sampling in high dimensions.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::