Probabilistic Joint and Individual Variation Explained (ProJIVE) for Data Integration
| Authors | Raphiel J. Murden et al. |
| Year | 2026 |
| Field | Statistics / ML |
| arXiv | 2603.12351 |
| Download | |
| Categories | stat.ML, cs.LG, stat.CO, stat.ME |
Abstract
Collecting multiple types of data on the same set of subjects is common in modern scientific applications including, genomics, metabolomics, and neuroimaging. Joint and Individual Variance Explained (JIVE) seeks a low-rank approximation of the joint variation between two or more sets of features captured on common subjects and isolates this variation from that unique to eachset of features. We develop an expectation-maximization (EM) algorithm to estimate a probabilistic model for the JIVE framework. The model extends probabilistic principal components analysis to multiple data sets. Our maximum likelihood approach simultaneously estimates joint and individual components, which can lead to greater accuracy compared to other methods. We apply ProJIVE to measures of brain morphometry and cognition in Alzheimer's disease. ProJIVE learns biologically meaningful courses of variation, and the joint morphometry and cognition subject scores are strongly related to more expensive existing biomarkers. Data used in preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Code to reproduce the analysis is available on our GitHub page.
Engineering Breakdown
Plain English
This paper develops ProJIVE, a probabilistic machine learning method for analyzing multiple types of data collected on the same subjects—like brain imaging and cognitive test scores together. The key problem is separating the variation that two datasets share (joint variation) from the variation unique to each dataset (individual variation), which is crucial for understanding relationships in genomics, metabolomics, and neuroimaging studies. The authors build an expectation-maximization (EM) algorithm that extends probabilistic PCA to handle multiple datasets simultaneously, estimating joint and individual components through maximum likelihood optimization. This approach delivers greater accuracy than existing methods because it jointly optimizes all components rather than solving them sequentially.
Core Technical Contribution
The core novelty is formulating JIVE as a probabilistic generative model solved via EM, rather than treating it as a purely deterministic decomposition problem. Traditional JIVE uses alternating least squares or other non-probabilistic methods; this work reframes it in a Bayesian framework where you have explicit latent variables for joint and individual variation, noise distributions, and can derive principled maximum likelihood estimates. This probabilistic formulation enables proper uncertainty quantification and statistical inference on the components, which prior deterministic methods cannot provide. By optimizing all parameters jointly through EM rather than sequentially, the method avoids local minima that plague greedy decomposition approaches.
How It Works
The input is multiple data matrices (e.g., X₁ for brain morphometry, X₂ for cognitive scores) sharing the same N subjects but with different numbers of features. The model assumes each data matrix decomposes into three components: a shared low-rank joint structure (J), individual structure unique to that dataset (Iₖ), and Gaussian noise (E). The EM algorithm iteratively runs expectation steps that compute posterior distributions over the latent joint and individual components given current parameter estimates, then maximization steps that update the loading matrices and variance parameters to maximize likelihood. During E-steps, the algorithm infers the unobserved low-dimensional representations; during M-steps, it optimizes the high-dimensional loadings and noise parameters. This jointly estimates all components across all datasets, avoiding the sequential bias of alternating least squares methods used in classical JIVE.
Production Impact
For production systems analyzing multi-modal medical or biological data, this approach enables automated, principled separation of shared and unique signal patterns without manual tuning. If you're building a genomics pipeline that integrates genetic data, gene expression, and protein data from the same patients, ProJIVE could identify which variations are consistent across all three modalities (likely causal) versus which are measurement-specific artifacts. The EM algorithm is computationally more expensive than greedy alternatives—each iteration requires matrix operations scaling with O(p₁p₂N) where p are feature counts and N is samples—so for ultra-high-dimensional data (millions of features), you'd need careful implementation or approximations. Integration is straightforward: it fits into standard ML pipelines as a preprocessing/dimensionality-reduction step, outputting low-rank joint and individual component matrices that feed downstream models. The probabilistic nature means you get principled confidence intervals on discovered components, valuable for clinical or scientific validation where uncertainty matters.
Limitations and When Not to Use This
The paper assumes Gaussian noise and low-rank structure, which fails when data contains heavy-tailed outliers, discrete values, or non-linear relationships—common in count data from sequencing or categorical clinical variables. Computational complexity and convergence properties aren't thoroughly analyzed; the EM algorithm may require many iterations or get stuck in local maxima with high-dimensional data, and the paper doesn't specify stopping criteria or scalability limits. The method requires you to specify the rank (number of latent dimensions) for the joint and individual components beforehand; incorrect rank selection severely degrades performance, and the paper doesn't provide principled rank-selection heuristics beyond cross-validation. The abstract cuts off at the application section, so it's unclear whether ProJIVE was actually validated on real brain morphometry data or what performance gains were quantified—missing experimental results limit confidence in practical utility.
Research Context
This work extends the JIVE framework (originally by O'Connell et al., ~2016) from its deterministic matrix-factorization origins into the probabilistic modeling paradigm, aligning it with modern statistical ML practice. It builds directly on probabilistic PCA (Tipping & Bishop, 1999) but extends it to the multi-table setting, which is relevant to the growing intersection of multi-omics and neuroimaging research. The probabilistic treatment opens the door to hierarchical Bayesian extensions, model selection via marginal likelihood, and integration with downstream generative models—directions that deterministic JIVE cannot pursue. This likely influences future work on joint modeling of multi-modal medical data, particularly in settings where uncertainty quantification and principled inference are regulatory or scientific requirements.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::