Who Guards the Guardians? The Challenges of Evaluating Identifiability of Learned Representations
| Authors | Shruti Joshi et al. |
| Year | 2026 |
| Field | Machine Learning |
| arXiv | 2602.24278 |
| Download | |
| Categories | cs.LG |
Abstract
Identifiability in representation learning is commonly evaluated using standard metrics (e.g., MCC, DCI, R^2) on synthetic benchmarks with known ground-truth factors. These metrics are assumed to reflect recovery up to the equivalence class guaranteed by identifiability theory. We show that this assumption holds only under specific structural conditions: each metric implicitly encodes assumptions about both the data-generating process (DGP) and the encoder. When these assumptions are violated, metrics become misspecified and can produce systematic false positives and false negatives. Such failures occur both within classical identifiability regimes and in post-hoc settings where identifiability is most needed. We introduce a taxonomy separating DGP assumptions from encoder geometry, use it to characterise the validity domains of existing metrics, and release an evaluation suite for reproducible stress testing and comparison.
Engineering Breakdown
Plain English
This paper exposes a critical blind spot in how we evaluate whether neural networks learn identifiable representations—the ability to recover ground-truth factors from learned embeddings. Researchers show that standard evaluation metrics (MCC, DCI, R²) used on synthetic benchmarks don't actually measure what we think they measure; they produce systematic false positives and false negatives when their implicit assumptions about the data-generating process or encoder geometry are violated. The authors introduce a taxonomy distinguishing these two sources of assumption violations, showing that metric failures occur even within settings where identifiability theory guarantees recovery should work. This means production systems relying on these evaluations to validate representation learning may have false confidence in their models' interpretability.
Core Technical Contribution
The paper's core novelty is a systematic decomposition of why standard identifiability metrics fail, separating failures into two independent categories: violations of data-generating process (DGP) assumptions versus violations of encoder geometry assumptions. Rather than proposing a new metric, the authors provide a diagnostic framework—a taxonomy—that allows practitioners to understand why a metric is misleading in their specific setting. They demonstrate that metric misspecification is not rare edge-case behavior but occurs systematically both in classical identifiability regimes (where theory says recovery should work) and in post-hoc settings (where identifiability is most practically needed). This taxonomic approach is novel because prior work treated metric failures as isolated problems rather than consequences of violated structural assumptions.
How It Works
The paper's methodology starts by taking existing identifiability metrics (MCC, DCI, R²) and decomposing the assumptions each one encodes about how data is generated and how the encoder transforms it. For each metric, the authors construct synthetic experiments that selectively violate individual assumptions while keeping others constant, allowing them to isolate which assumption violation causes which failure mode. They then categorize failures into two bins: (1) DGP assumption violations—when the actual data doesn't match the distribution the metric assumes (e.g., assuming independent factors when they're correlated), and (2) encoder geometry violations—when the learned representation has structural properties the metric doesn't account for (e.g., rotational symmetry, scaling variations). By systematically applying this framework to benchmarks with known ground truth, they map which metric-DGP-encoder combinations produce false positives (claiming identifiability when it hasn't occurred) versus false negatives (missing genuine identifiability). The output is a practical decision tree showing which metric is reliable under which conditions.
Production Impact
For teams training representation models, this work means you can no longer trust off-the-shelf identifiability metric scores at face value—you must first verify that your data and encoder match the metric's underlying assumptions. This adds a validation step to any pipeline claiming to learn interpretable or disentangled representations: before deploying a model based on a high DCI or MCC score, you should check whether your encoder has architectural properties (like tied weights or symmetries) that violate the metric's assumptions, and whether your data has statistical properties (like factor correlations) that do the same. The practical cost is moderate: it requires generating synthetic variants of your data with controlled properties and running ablation studies on your encoder architecture, adding perhaps 2-3 days of engineering work per model. The payoff is avoiding false confidence in interpretability claims that could lead to downstream errors in domains like medical AI or autonomous systems where understanding model decisions is critical.
Limitations and When Not to Use This
The paper does not propose solutions for making metrics robust to assumption violations—it only diagnoses why they fail. The taxonomy itself requires practitioners to manually check assumptions for their specific DGP and encoder, which is not fully automated and may miss novel assumption types the paper doesn't enumerate. The work is also limited to synthetic benchmarks with known ground-truth factors; real-world data where ground truth is unknown may have assumption violations the taxonomy doesn't account for. Additionally, the paper doesn't address whether any single metric can be made universally robust or whether practitioners will always need to maintain multiple metrics for different use cases. Finally, the post-hoc identifiability regime (where you're trying to identify factors in already-trained models) may present failure modes beyond the DGP/encoder dichotomy that the paper's framework doesn't fully capture.
Research Context
This work builds on the identifiability theory literature in representation learning (following frameworks like β-VAE, Factor-VAE, and formal identifiability guarantees by Kumar et al.), but shifts focus from theory to evaluation. It sits at the intersection of two communities: the representation learning community that has adopted metrics like DCI and MCC as standards, and the interpretability/explainability community concerned with whether learned representations actually recover meaningful factors. The paper opens a research direction around metric auditing—systematically checking that evaluation procedures measure what they claim to measure—which parallels similar work in fairness and robustness where standard metrics have been shown to be misspecified. It also motivates future work on assumption-aware metrics that explicitly encode their constraints or adaptive metrics that detect and correct for assumption violations.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::