Explainable cluster analysis: a bagging approach
| Authors | Federico Maria Quetti et al. |
| Year | 2026 |
| Field | Statistics / ML |
| arXiv | 2603.19840 |
| Download | |
| Categories | stat.ML, cs.LG |
Abstract
A major limitation of clustering approaches is their lack of explainability: methods rarely provide insight into which features drive the grouping of similar observations. To address this limitation, we propose an ensemble-based clustering framework that integrates bagging and feature dropout to generate feature importance scores, in analogy with feature importance mechanisms in supervised random forests. By leveraging multiple bootstrap resampling schemes and aggregating the resulting partitions, the method improves stability and robustness of the cluster definition, particularly in small-sample or noisy settings. Feature importance is assessed through an information-theoretic approach: at each step, the mutual information between each feature and the estimated cluster labels is computed and weighted by a measure of clustering validity to emphasize well-formed partitions, before being aggregated into a final score. The method outputs both a consensus partition and a corresponding measure of feature importance, enabling a unified interpretation of clustering structure and variable relevance. Its effectiveness is demonstrated on multiple simulated and real-world datasets.
Engineering Breakdown
Plain English
This paper addresses a fundamental problem in clustering: most clustering algorithms produce cluster assignments but don't explain which features actually drove the grouping. The authors propose an ensemble approach using bagging (bootstrap aggregating) combined with feature dropout to assign importance scores to each feature, similar to how random forests work in supervised learning. The method resamples data multiple times, fits clustering models with different feature subsets, and measures how much each feature contributes to cluster formation using mutual information between features and cluster labels. The result is a more stable, robust clustering solution that also tells you which features matter most—particularly valuable in small-sample or high-noise scenarios where standard clustering is unreliable.
Core Technical Contribution
The core novelty is applying ensemble bagging techniques and feature importance mechanisms from supervised learning to unsupervised clustering. Specifically, the authors combine bootstrap resampling of observations with feature dropout (randomly excluding features during resampling steps) and aggregate the resulting cluster partitions to stabilize cluster definitions. They then compute feature importance using information-theoretic measures—mutual information between each feature and the discovered cluster labels—creating a principled way to rank which features drive the clustering structure. This is non-trivial because feature importance in unsupervised learning is harder to define than in supervised settings where you have ground-truth labels; mutual information provides a model-agnostic solution that works with any clustering algorithm.
How It Works
The pipeline works as follows: start with your data matrix X and choose a base clustering algorithm (e.g., K-means). In the ensemble loop, repeatedly: (1) draw a bootstrap sample (random sample with replacement) from your observations, (2) randomly select a subset of features (dropout), (3) run the clustering algorithm on this resampled subset, recording the cluster assignments. After B iterations (e.g., 100-500 bags), you aggregate all B partitions—typically by building a consensus partition or taking a majority vote on which observations cluster together. For feature importance, compute the mutual information I(Feature_j; Cluster_Labels) for each feature j by measuring how much knowing feature j's values reduces uncertainty about cluster membership. High MI means that feature strongly distinguishes between clusters; low MI means it's noise. The final output is both a refined cluster partition and a ranked list of feature importances.
Production Impact
For engineers building clustering pipelines, this approach solves two concrete problems: (1) cluster instability in small-sample or noisy data—bagging naturally reduces variance in partition estimates, improving reproducibility across runs, and (2) interpretability—business stakeholders and data scientists can now see which features justify the clustering, enabling trust and debugging. In production, you'd integrate this by replacing your single clustering call with an ensemble loop (modest compute overhead: ~3–10x depending on ensemble size, easily parallelized across cores), then exposing feature importances in your output schema alongside cluster assignments. The trade-off is latency: instead of one clustering run you run B of them, though each individual run is on a smaller feature subset so wall-clock time can be acceptable. Integration is straightforward if your clustering library is modular; you wrap it with bagging and MI computation logic. Data requirements are minimal—no labeled data needed, works with unsupervised data as-is.
Limitations and When Not to Use This
This approach assumes that mutual information is the right measure of feature importance, which may not capture non-linear or higher-order feature interactions that drive clustering; MI is pairwise and treats each feature independently. The method also requires choosing hyperparameters (number of bags B, feature dropout rate, clustering algorithm and its parameters) without clear guidance in the abstract, and poor choices can lead to meaningless importance scores or cluster drift. Computational cost scales linearly with B, making it expensive for very large datasets or high-dimensional data with many bags; the paper doesn't address scalability to millions of observations. Additionally, the approach assumes that cluster labels are meaningful and stable to begin with—if the underlying data has no natural clustering structure, bagging and MI computation won't rescue that; the method is orthogonal to determining optimal cluster numbers. Follow-up work should include guidance on hyperparameter selection, comparison with other explainability methods (e.g., LIME for clusters), and handling of mixed data types or categorical features.
Research Context
This work bridges explainability (a growing priority across ML) with unsupervised learning, building on two established ideas: ensemble methods and feature importance from random forests (Breiman et al.) applied to a new domain. It extends prior work on clustering stability (e.g., stability selection, consensus clustering) by adding the interpretability dimension, and it contributes to the broader trend of 'explainable clustering' which has seen recent work on attention-based clustering and saliency maps for cluster assignments. The research opens up questions about how to formalize and measure explainability in unsupervised settings, where ground truth is absent—moving beyond accuracy to include interpretability as a first-class objective. This likely influences future work on other unsupervised tasks (dimensionality reduction, anomaly detection) where practitioners also want to understand what the model learned.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::