Structure-Preserving Multi-View Embedding Using Gromov-Wasserstein Optimal Transport
| Authors | Rafael Pereira Eufrazio et al. |
| Year | 2026 |
| Field | Statistics / ML |
| arXiv | 2604.02610 |
| Download | |
| Categories | stat.ML, cs.LG |
Abstract
Multi-view data analysis seeks to integrate multiple representations of the same samples in order to recover a coherent low-dimensional structure. Classical approaches often rely on feature concatenation or explicit alignment assumptions, which become restrictive under heterogeneous geometries or nonlinear distortions. In this work, we propose two geometry-aware multi-view embedding strategies grounded in Gromov-Wasserstein (GW) optimal transport. The first, termed Mean-GWMDS, aggregates view-specific relational information by averaging distance matrices and applying GW-based multidimensional scaling to obtain a representative embedding. The second strategy, referred to as Multi-GWMDS, adopts a selection-based paradigm in which multiple geometry-consistent candidate embeddings are generated via GW-based alignment and a representative embedding is selected. Experiments on synthetic manifolds and real-world datasets show that the proposed methods effectively preserve intrinsic relational structure across views. These results highlight GW-based approaches as a flexible and principled framework for multi-view representation learning.
Engineering Breakdown
Plain English
This paper addresses the multi-view learning problem—integrating multiple different representations of the same data samples to find a shared low-dimensional embedding. Traditional approaches either concatenate features or assume explicit alignment between views, which breaks down when the data has different geometric structures or nonlinear distortions across views. The authors propose two new methods based on Gromov-Wasserstein optimal transport: Mean-GWMDS, which averages distance matrices across views before embedding, and Multi-GWMDS, which generates multiple candidate embeddings and selects geometry-consistent ones. This approach avoids forcing artificial alignment and instead respects the intrinsic geometry of each view while finding coherent shared structure.
Core Technical Contribution
The key innovation is applying Gromov-Wasserstein optimal transport—a distance metric that compares geometric structures rather than raw coordinates—to multi-view embedding for the first time. Instead of requiring views to live in the same space or be explicitly aligned, GW-based methods compare the relational geometry (distance patterns) within each view and find an embedding that preserves these relationships. Mean-GWMDS aggregates relational information by averaging distance matrices then solving a GW-based multidimensional scaling problem. Multi-GWMDS goes further by generating multiple candidate embeddings and selecting among them based on consistency with the original view geometries, avoiding information loss from naive averaging.
How It Works
The input is K different view-specific representations of the same N samples (e.g., images and text descriptions of products). For each view k, compute the pairwise distance matrix D_k ∈ ℝ^(N×N), capturing how similar samples are within that view's geometry. Mean-GWMDS: average these distance matrices element-wise to get D_mean, then apply multidimensional scaling (MDS) optimized using the Gromov-Wasserstein distance metric, which measures how well the low-dimensional embedding preserves the relational structure of D_mean. Multi-GWMDS: instead of averaging, generate multiple candidate embeddings (e.g., from each view independently or from subsets), then select the candidate that minimizes GW distance to all original view geometries simultaneously, effectively choosing the embedding most consistent with the actual data. The output is a single N×d low-dimensional embedding that respects the geometry of all input views.
Production Impact
This approach solves a real problem in multi-modal ML systems: fusing data from different sensors, modalities, or feature spaces without losing information to forced alignment. In a production recommendation system using both user text reviews and image data, naive concatenation assumes both modalities are equally important and directly comparable; this method instead learns what structure matters in each modality separately then finds the shared signal. Computationally, you pay the cost of: (1) computing pairwise distance matrices (O(N²) per view, manageable for N < 100K), (2) solving a GW-based optimization problem (typically via Sinkhorn iterations, ~O(N³) worst-case but often faster in practice), and (3) for Multi-GWMDS, generating and comparing multiple candidates (adds a constant factor overhead). For real-time systems, pre-compute embeddings offline; for streaming data, retraining becomes expensive. The approach works best when you have 2–5 views; with many views, distance matrix averaging may oversimplify.
Limitations and When Not to Use This
The method assumes all views represent the same underlying samples (no missing views or outlier samples), which often fails in real federated or heterogeneous data scenarios. Computational cost scales poorly with sample count—distance matrix computation is O(N²) in memory and time, making it infeasible for N > 1M without sampling or approximate methods not discussed here. The paper truncates at 'generate,' so the full Multi-GWMDS algorithm and its convergence properties are unclear; without complete details, reproducibility and theoretical guarantees are uncertain. GW distance computation itself is sensitive to hyperparameters (entropic regularization strength, number of Sinkhorn iterations) and the paper likely doesn't provide guidance on setting these for different data types. Finally, there's no analysis of what happens when view geometries are fundamentally conflicting (e.g., views that disagree on sample similarity), or when one view is extremely noisy—the method may produce a meaningless "compromise" embedding.
Research Context
This work builds on a decade of optimal transport advances in machine learning, particularly the Gromov-Wasserstein distance (Peyré et al., 2016) which has been applied to domain adaptation and generative modeling but rarely to multi-view learning in this way. Multi-view learning itself is a classical problem (surveys by Xu et al., 2013), with prior methods using CCA (canonical correlation analysis), co-training, or explicit graph alignment. The novelty is replacing algebraic assumptions (like linear correlation in CCA) with geometric ones (preserving distance structure via OT). This opens a research direction toward optimal-transport-based multi-modal fusion, potentially leading to follow-up work on scalable approximations, robustness to missing or corrupt views, and applications to high-dimensional medical imaging (multi-modal MRI/CT) or multi-lingual embeddings where geometric rather than algebraic fusion might be more natural.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::