Kernel Integrated $R^2$: A Measure of Dependence

Authors	Pouya Roudaki et al.
Year	2026
Field	Statistics / ML
arXiv	2602.22985
PDF	Download
Categories	stat.ML, cs.IT, cs.LG

Abstract

We introduce kernel integrated $R^2$, a new measure of statistical dependence that combines the local normalization principle of the recently introduced integrated $R^2$ with the flexibility of reproducing kernel Hilbert spaces (RKHSs). The proposed measure extends integrated $R^2$ from scalar responses to responses taking values on general spaces equipped with a characteristic kernel, allowing to measure dependence of multivariate, functional, and structured data, while remaining sensitive to tail behaviour and oscillatory dependence structures. We establish that (i) this new measure takes values in $[0,1]$, (ii) equals zero if and only if independence holds, and (iii) equals one if and only if the response is almost surely a measurable function of the covariates. Two estimators are proposed: a graph-based method using $K$-nearest neighbours and an RKHS-based method built on conditional mean embeddings. We prove consistency and derive convergence rates for the graph-based estimator, showing its adaptation to intrinsic dimensionality. Numerical experiments on simulated data and a real data experiment in the context of dependency testing for media annotations demonstrate competitive power against state-of-the-art dependence measures, particularly in settings involving non-linear and structured relationships.

---

Engineering Breakdown

Plain English

This paper introduces kernel integrated R², a new statistical measure that quantifies dependence between variables and their predictors, extending beyond scalar outputs to handle multivariate, functional, and structured data. The measure combines local normalization principles with reproducing kernel Hilbert spaces (RKHSs) to remain sensitive to both tail behavior and oscillatory patterns in data. It guarantees three key properties: values always fall between 0 and 1, equals zero exactly when variables are independent, and equals one exactly when the response is a deterministic function of the covariates. Two estimators are proposed to compute this measure in practice.

Core Technical Contribution

The core novelty is extending the recently-introduced integrated R² metric from scalar-valued responses to general structured spaces via kernel methods. Previous dependence measures like mutual information or distance correlation struggle with computational scalability or lack local interpretability; this work leverages RKHS theory to maintain the local normalization principle while gaining flexibility for complex data types. The technical innovation is showing how to reformulate dependence through a characteristic kernel, enabling measurement of structured outputs (functions, multivariate distributions, manifold-valued data) while preserving sensitivity to non-monotonic relationships. The authors prove that under mild conditions, their measure satisfies the three desired theoretical properties, making it both principled and practical.

How It Works

The method starts with a response variable Y taking values in a general space and covariates X; the goal is to measure how much variation in Y is explained by X. A characteristic kernel K is chosen for the response space—this kernel implicitly defines a reproducing kernel Hilbert space (RKHS) where functions can be evaluated and compared. The integrated R² principle applies local normalization: at each covariate value x, you compute the conditional variance of Y given X=x in the RKHS norm, then average across the covariate distribution, dividing by total variance to get a normalized measure. The estimators convert this theoretical quantity into computable statistics from finite samples, using kernel matrix operations and empirical conditional expectations. The output is a single scalar in [0,1] that quantifies dependence regardless of whether Y is univariate, multivariate, functional (infinite-dimensional), or has other exotic structure.

Production Impact

In production ML systems, this measure enables engineers to quantify feature importance and explanatory power for complex output spaces—a major gap in current tooling. For example, in time-series forecasting where outputs are functional (entire future trajectories), or in structured prediction (graphs, sequences, images), existing R² metrics don't apply; kernel integrated R² fills this gap with a theoretically-grounded alternative to post-hoc feature attribution methods. Teams could use this during feature engineering to identify which input variables truly drive predictions for complex outputs, reducing feature dimensionality and improving model interpretability. The trade-off is computational: computing kernel matrices scales as O(n²) or O(n³) depending on the estimator, so for datasets larger than ~100k samples, practitioners need approximations (Nyström methods, random features) to keep inference tractable. Integration into existing pipelines requires choosing an appropriate kernel for your output space—a domain-specific decision that requires some statistical knowledge.

Limitations and When Not to Use This

The paper assumes the existence of a characteristic kernel for the response space, but choosing this kernel in practice is non-trivial and can heavily influence results; no principled guidance is given for practitioners unfamiliar with kernel methods. Computational cost grows polynomially with sample size, making the method impractical for very large datasets without additional approximations not discussed in the abstract. The measure assumes the covariate distribution is well-behaved (bounded support or fast-decaying tails) since the normalization depends on marginal variance estimation; extreme outliers or highly heavy-tailed X could destabilize estimates. The estimators' finite-sample properties (convergence rates, bias) are not detailed in the abstract, so it's unclear how much data is needed for reliable inference on realistic problems.

Research Context

This work builds directly on the recent integrated R² framework, which introduced the local normalization principle as an alternative to global dependence measures; the authors show how to extend that idea to modern ML settings with complex output spaces. It relates to a broader family of kernel-based dependence measures (kernel CCA, distance covariance, maximum mean discrepancy) but differs by focusing on directional dependence (Y given X) rather than mutual dependence, and by maintaining interpretability via local normalization. The paper advances the theory of dependence measures for structured data, a research direction gaining importance as ML tackles functional outputs, graph-structured predictions, and manifold-valued regression. The result opens avenues for new conditional independence tests, fairness auditing tools, and explanability methods tailored to non-Euclidean outputs.

---

:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::

---

Back to Research Lab → · Subscribe to AI Letters →