Concentration and Calibration in Predictive Bayesian Inference
| Authors | David T. Frazier & Hui Wang |
| Year | 2026 |
| Field | AI / ML |
| arXiv | 2605.00455 |
| Download | |
| Categories | stat.ME, stat.ML |
Abstract
Predictive Bayesian inference (PBI) represents a model-and prior-agnostic approach to standard Bayesian inference which allows users to quantify uncertainty for a functional of interest only by specifying a forward predictive model for future unobserved data. The flexibility and generality of this framework have led to a host of novel algorithms for implementing this approach, and many empirical applications, yet the reliability of the resulting inferences for the underlying statistical functional of interest remains unclear. Herein, we demonstrate that when using PBI for a population functional of interest, the resulting posterior concentrates onto a well-defined quantity that explicitly depends on the forward predictive model used to implement the predictive recursion underlying the method. Furthermore, the forward predictive model entirely determines the uncertainty quantification produced in PBI. Consequently, our results show that if the predictive model does not capture all relevant features of the data, and, even in very simple examples, the coverage of predictive Bayes credible sets for the population value of the functional of interest can be arbitrarily close to zero. We carefully explain why this occurs, and show that this behavior is directly tied to the inaccuracy of the forward predictive model used to produce future observations within the PBI framework. As a consequence, our results imply that in order for PBI to deliver calibrated posterior inferences, the resulting predictive engine used to generate posterior samples must contain, in a well-defined sense, the true DGP, else inferences generated under this framework will not be calibrated.
Engineering Breakdown
Plain English
This paper addresses a fundamental problem in predictive Bayesian inference (PBI): when you build a Bayesian model to predict unobserved future data and use it to make inferences about a quantity of interest, what exactly does your posterior distribution actually converge to? The authors prove that the posterior concentrates on a well-defined target that explicitly depends on your forward predictive model — meaning the specific model you chose matters significantly for what you're actually learning about. This is important because PBI has become popular for its flexibility (you only need to specify a forward model, not a full likelihood), but practitioners haven't had clarity on whether their inferences are reliable. The paper provides theoretical guarantees showing that PBI posteriors do concentrate meaningfully, but with an important caveat: the target depends on your modeling choices in non-obvious ways.
Core Technical Contribution
The core contribution is establishing concentration theory for predictive Bayesian inference when inferring population-level functionals. Prior work showed PBI was flexible and empirically useful, but lacked formal guarantees about what the posterior actually learns — it could concentrate on the wrong quantity if the forward predictive model was misspecified. The authors prove that the posterior concentrates on a specific functional of the true data-generating process, and crucially, this target is model-dependent: it's determined by both the true distribution and your choice of forward predictive model. This creates an explicit link between model selection and inference target, filling a theoretical gap that was preventing practitioners from understanding when PBI results are trustworthy. The theoretical framework also naturally leads to diagnostics (like calibration checks) to assess whether your PBI inferences are reliable for your specific application.
How It Works
Predictive Bayesian inference works by specifying a forward model p(y_future | data, θ) that predicts what future data will look like, rather than specifying a likelihood for current observations. You then use a predictive recursion algorithm that iteratively updates beliefs about parameters θ using this forward model as data arrives. The paper's key technical contribution is proving that as you collect more data, the posterior distribution p(θ | observed data) concentrates around a limit that depends on three things: (1) the true data-generating process, (2) your forward predictive model choice, and (3) the functional of interest (e.g., a population mean). The concentration happens at a rate determined by the KL divergence between your forward model and the true distribution. The authors show this concentration is explicit and measurable, meaning you can compute what quantity your posterior is actually learning about and check if it matches your target quantity through calibration diagnostics — comparing the posterior's predicted distribution against empirical frequencies.
Production Impact
For production systems, this paper provides a much-needed theoretical foundation for using PBI in decision-critical applications. If you're using PBI (e.g., for epidemiological forecasting, causal inference without specifying a full likelihood, or learning from simulator-based models), you can now formally check whether your posterior is learning about the right quantity using calibration diagnostics. The framework tells you exactly what your posterior converges to, so you can detect when model misspecification causes your inferences to target the wrong quantity — before making bad decisions. Implementation-wise, the paper suggests adding concentration and calibration checks to your PBI pipeline: compute what functional your model should concentrate on theoretically, run your inference, then verify that empirical frequencies match posterior predictions. The trade-off is computational: checking calibration requires holdout data and multiple posterior samples, adding 10-20% overhead to a typical pipeline, but this is justified when inference reliability is costly (healthcare, finance). The main practical win is diagnosing when 'your model is flexible but wrong' — PBI will still give you a confident posterior, just not about what you think.
Limitations and When Not to Use This
The paper assumes the forward predictive model is tractable enough to implement the predictive recursion — if your simulator is expensive or high-dimensional, the practical algorithms become slow even if theory says concentration happens. The concentration results are asymptotic (as data → ∞), but provide no finite-sample rates or guidance on how much data you actually need, so small-data regimes remain opaque. The paper focuses on population functionals (e.g., means, quantiles), but real applications often care about individual predictions or decision-making under the posterior, where concentration to the 'right' quantity doesn't guarantee good decisions if model bias is large. Most critically, the paper proves that the posterior concentrates to a model-dependent target, but doesn't provide algorithms for choosing models to concentrate on your true target of interest — if all your candidate models are misspecified, you'll reliably learn about the wrong thing. Finally, the calibration diagnostics require held-out test data and assume the test distribution matches your future application distribution, which is often violated in practice.
Research Context
This paper builds on recent work in model-agnostic Bayesian inference and extends classical concentration theory (Berry-Esseen bounds, Bernstein-von Mises theory) to the PBI setting where the model generating your inferences and the true data-generating process can differ substantially. It connects to the broader literature on predictive modeling and misspecified models in Bayesian inference, where research has shown that Bayesian posteriors under misspecification don't always converge to the truth — but prior work focused on the case where you misspecify a full likelihood, not the flexibly-specified forward model case. The calibration framework it proposes relates to recent work on posterior predictive checks and out-of-sample validation for Bayesian models. This opens up research directions in (1) finite-sample concentration rates for PBI, (2) model selection theory that explicitly targets concentration on user-relevant functionals rather than likelihood maximization, and (3) extensions to high-dimensional functionals and online/streaming settings where re-fitting models at each step is expensive.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::