Hierarchical Inference and Closure Learning via Adaptive Surrogates for ODEs and PDEs
| Authors | Pengyu Zhang et al. |
| Year | 2026 |
| Field | Machine Learning |
| arXiv | 2603.03922 |
| Download | |
| Categories | cs.LG, stat.ML |
Abstract
Inverse problems are the task of calibrating models to match data. They play a pivotal role in diverse engineering applications by allowing practitioners to align models with reality. In many applications, engineers and scientists do not have a complete picture of i) the detailed properties of a system (such as material properties, geometry, initial conditions, etc.); ii) the complete laws describing all dynamics at play (such as friction laws, complicated damping phenomena, and general nonlinear interactions). In this paper, we develop a principled methodology for leveraging data from collections of distinct yet related physical systems to jointly estimate the individual model parameters of each system, and learn the shared unknown dynamics in the form of an ML-based closure model. To robustly infer the unknown parameters for each system, we employ a hierarchical Bayesian framework, which allows for the joint inference of multiple systems and their population-level statistics. To learn the closures, we use a maximum marginal likelihood estimate of a neural network embeded within the ODE/PDE formulation of the problem. To realize this framework we utilize the ensemble Metropolis-Adjusted Langevin Algorithm (MALA) for stable and efficient sampling. To mitigate the computational bottleneck of repetitive forward evaluations in solving inverse problems, we introduce a bilevel optimization strategy to simultaneously train a surrogate forward model alongside the inference. Within this framework, we evaluate and compare distinct surrogate architectures, specifically Fourier Neural Operators (FNO) and parametric Physics-Informed Neural Network (PINNs).
Engineering Breakdown
Plain English
This paper addresses a common engineering challenge: when you have multiple related physical systems with incomplete knowledge of their individual parameters and shared underlying dynamics, how do you calibrate them all simultaneously using data? The authors propose a methodology that jointly estimates system-specific parameters across multiple experiments while learning a shared ML-based closure model that captures unknown physics. Instead of tuning each system in isolation, they leverage relationships between systems to improve parameter estimation and discover missing dynamics that govern all of them. This is particularly valuable in domains like materials science, fluid dynamics, or mechanical systems where you have partial differential equations but don't fully know material properties, boundary conditions, or nonlinear interaction terms.
Core Technical Contribution
The core novelty is a principled framework for multi-system inverse problems that decouples system-specific parameters from shared unknown dynamics. Rather than treating each inverse problem independently, the method recognizes that collections of related systems share common physics (the unknown dynamics) while differing only in their individual configuration parameters—and it exploits this structure to improve both parameter recovery and dynamics learning. The closure model is learned jointly via an ML component that operates alongside traditional physics-based optimization, creating a hybrid approach where data from one system informs the discovery of shared laws applicable to all systems. This is fundamentally different from prior work that either solves single-system inverse problems or requires a pre-specified dynamics model; here the shared physics emerges from the data itself.
How It Works
The methodology takes multiple datasets from distinct but related physical systems as input, where each system has its own unknown parameters (material properties, geometry, initial conditions) but operates under shared unknown dynamics (closure terms, interaction laws). For each system i, the framework maintains a parameter vector θ_i and jointly optimizes across all systems to minimize a loss function combining prediction error and regularization terms. The shared unknown dynamics are represented as an ML closure model (typically a neural network) that operates on state variables and outputs corrections to the physics-based equations—this closure model is shared across all systems and learned from collective data. The optimization alternates between: (1) refining individual system parameters θ_i to match their specific observations, and (2) updating the closure model weights to better capture shared physics that explains residuals across all systems simultaneously. The output is a set of calibrated parameters for each system plus a trained closure model that generalizes to new systems with similar dynamics.
Production Impact
For engineers calibrating complex physical systems in manufacturing, materials processing, or structural analysis, this approach eliminates the need to solve inverse problems sequentially or in isolation. Instead of running separate expensive calibration experiments for each variant of a system, practitioners can pool data from multiple similar systems and obtain better parameter estimates with less total data—this directly reduces experimental costs and iteration time. The learned closure model provides a reusable component that can be transferred to new systems with the same underlying physics, accelerating deployment of digital twins or surrogate models. However, adoption requires: (1) careful multi-system experimental design to ensure systems are truly related, (2) sufficient diversity in system parameters so the model doesn't overfit to one configuration, and (3) integration of an ML component into what may be a traditional physics simulation pipeline, adding complexity to deployment and requiring validation that the learned closure is physically plausible across unseen parameter ranges.
Limitations and When Not to Use This
The paper assumes that multiple systems truly share the same unknown dynamics—if different systems operate under fundamentally different physics, the approach breaks down and forces the closure model to learn a piecewise or mixture model it may not represent well. The methodology is sensitive to the choice of closure model architecture and the balance between data-fitting and regularization; poorly tuned trade-offs can lead to overfitting to the training systems or under-learning the true dynamics. The approach requires sufficient quantity and quality of data from multiple systems, which may be expensive to obtain in some domains; sparse or noisy data can degrade both parameter recovery and closure learning. Additionally, the paper's abstract cuts off and lacks details on computational complexity, convergence guarantees, and how the method scales when the number of systems or dimensionality of parameters grows large—these practical engineering concerns likely require reading the full paper.
Research Context
This work builds on decades of inverse problem research in computational mechanics and data assimilation, but extends it from single-system calibration (classical parameter estimation) to the multi-system regime where structure in the problem can be exploited. It relates to the broader field of physics-informed machine learning, where neural networks learn corrections or closure models for incomplete physics—papers like those on neural operator learning and physics-informed neural networks (PINNs) share the goal of discovering unknown dynamics. The contribution fits into emerging work on transfer learning for physics and multi-task learning in scientific computing, where sharing information across related problems improves sample efficiency. This opens research directions in: (1) theoretical analysis of when and why multi-system calibration improves identifiability, (2) uncertainty quantification that tracks both parameter uncertainty and closure model uncertainty, and (3) domain generalization—understanding when a learned closure transfers to systems with significantly different parameters or initial conditions.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::