Graph-Informed Adversarial Modeling: Infimal Subadditivity of Interpolative Divergences
| Authors | Panagiota Birmpa & Eric Joseph Hall |
| Year | 2026 |
| Field | Statistics / ML |
| arXiv | 2603.20025 |
| Download | |
| Categories | stat.ML, cs.LG |
Abstract
We study adversarial learning when the target distribution factorizes according to a known Bayesian network. For interpolative divergences, including -divergences, we prove a new infimal subadditivity principle showing that, under suitable conditions, a global variational discrepancy is controlled by an average of family-level discrepancies aligned with the graph. In an additive regime, this surrogate is exact. This provides a variational justification for replacing a graph-agnostic GAN with a monolithic discriminator by a graph-informed GAN with localized family-level discriminators. The result does not require the optimizer itself to factorize according to the graph. We also obtain parallel results for integral probability metrics and proximal optimal transport divergences, identify natural discriminator classes for which the theory applies, and present experiments showing improved stability and structural recovery relative to graph-agnostic baselines.
Engineering Breakdown
Plain English
This paper solves a fundamental problem in adversarial learning: how to train GANs more efficiently when the target distribution has known structure (a Bayesian network). The authors prove that instead of using a single discriminator, you can decompose it into multiple smaller discriminators, one for each local factor in the graph, and still achieve the same or better results. The key finding is an 'infimal subadditivity principle' showing that the global error is controlled by averaging local errors—this is exact in additive regimes. This provides theoretical justification for replacing monolithic GANs with graph-aware variants that use localized discriminators.
Core Technical Contribution
The core novelty is a variational decomposition theorem for adversarial divergences under graphical model structure. Prior work treated GANs as black boxes indifferent to target distribution structure; this paper proves you can factorize the discriminator according to a Bayesian network's factors and maintain or improve convergence guarantees. The authors establish this for an important class of divergences (f-divergences, integral probability metrics, optimal transport), showing the decomposed version is not just a heuristic but theoretically justified. The crucial insight is that the optimizer itself does NOT need to factorize—only the discriminator architecture must respect the graph structure.
How It Works
The method starts with a known Bayesian network representing the target distribution p(x)—factorized as a product over cliques or factors. Instead of training a single discriminator D that maps from full observation space to a scalar, you train a set of local discriminators {D_i}, each operating on a subset of variables corresponding to one factor. During training, the global divergence (e.g., Jensen-Shannon or f-divergence) is upper-bounded by a weighted average of local divergences: D_global ≤ (1/|factors|) * Σ D_local_i. The generator still sees the full observation and updates against the averaged local discriminator signals. In the additive divergence case, this bound is tight (equality holds), meaning no information is lost by decomposition. The framework works with interpolative divergences including (f,Γ)-divergences and integral probability metrics.
Production Impact
For engineers building large-scale GANs, this directly reduces computational cost: instead of one expensive discriminator processing high-dimensional data, you train multiple smaller discriminators in parallel on subsets of variables—potentially 2-10x speedup depending on factor size and graph structure. If your target domain has known structure (e.g., image generation with object parts, financial data with sector dependencies, medical data with clinical factor hierarchies), you gain both computational efficiency and better generalization because each local discriminator can specialize. Integration is straightforward: replace your monolithic discriminator architecture with a factor-aware one, compute losses per factor, average them, and backprop—standard PyTorch/TensorFlow. Trade-offs: you need to specify the Bayesian network upfront (misspecification degrades benefits), and coordination overhead increases slightly with graph size; however, empirical results suggest these are minimal if structure is correct.
Limitations and When Not to Use This
The method requires knowing the target distribution's Bayesian network structure a priori—if the graph is misspecified or the true distribution doesn't factorize cleanly, benefits vanish and you may lose expressiveness. The paper assumes the target p(x) is fixed and known; it does NOT handle structure learning or adaptation if the environment shifts. The theoretical guarantees (infimal subadditivity) hold under 'suitable conditions' which the abstract notes but doesn't fully enumerate—the full characterization of when these conditions hold in practice needs clarification. Finally, the paper does not provide empirical benchmarks on real datasets; it's a theoretical result that requires validation on standard generative modeling tasks (ImageNet, text, etc.) before practitioners can confidently adopt it.
Research Context
This builds on a long line of work in graphical model-informed machine learning (e.g., structured variational inference, factor graphs) and recent advances in understanding GAN convergence via divergence decomposition. It extends prior results on integral probability metrics and optimal transport to the adversarial setting. The contribution fits the trend toward inductive bias: moving from generic deep learning to architectures that encode domain structure (like how Vision Transformers encode spatial structure). This opens research directions in automatic structure discovery for GANs, extension to hierarchical and continuous graphical models, and application to semi-supervised or conditional generation where partial observability aligns with the factor graph.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::