Structural interpretability in SVMs with truncated orthogonal polynomial kernels
| Authors | Víctor Soto-Larrosa et al. |
| Year | 2026 |
| Field | Statistics / ML |
| arXiv | 2604.15285 |
| Download | |
| Categories | stat.ML, cs.LG |
Abstract
We study post-training interpretability for Support Vector Machines (SVMs) built from truncated orthogonal polynomial kernels. Since the associated reproducing kernel Hilbert space is finite-dimensional and admits an explicit tensor-product orthonormal basis, the fitted decision function can be expanded exactly in intrinsic RKHS coordinates. This leads to Orthogonal Representation Contribution Analysis (ORCA), a diagnostic framework based on normalized Orthogonal Kernel Contribution (OKC) indices. These indices quantify how the squared RKHS norm of the classifier is distributed across interaction orders, total polynomial degrees, marginal coordinate effects, and pairwise contributions. The methodology is fully post-training and requires neither surrogate models nor retraining. We illustrate its diagnostic value on a synthetic double-spiral problem and on a real five-dimensional echocardiogram dataset. The results show that the proposed indices reveal structural aspects of model complexity that are not captured by predictive accuracy alone.
Engineering Breakdown
Plain English
This paper presents a new interpretability method called ORCA (Orthogonal Representation Contribution Analysis) for understanding how Support Vector Machines using truncated orthogonal polynomial kernels make decisions. The key insight is that because these kernels have finite-dimensional reproducing kernel Hilbert spaces (RKHS) with explicit orthonormal bases, the trained SVM decision function can be decomposed exactly into interpretable components. Rather than using surrogate models or retraining, the authors quantify how the classifier's squared RKHS norm is distributed across different interaction orders, polynomial degrees, marginal effects, and pairwise contributions—enabling transparent post-training diagnosis without computational overhead.
Core Technical Contribution
The core novelty is exploiting the mathematical structure of orthogonal polynomial kernels to enable exact, parameter-free decomposition of SVM decision functions into intrinsically interpretable coordinates. Unlike existing interpretability methods that require approximations, surrogate models, or retraining, ORCA directly expresses the fitted classifier as a sum of orthogonal components weighted by normalized contribution indices (OKC indices). This is the first work to systematically decompose polynomial kernel SVMs along interaction orders and marginal coordinate effects simultaneously, providing structured insight into feature interactions without any external model or data manipulation.
How It Works
The method starts with an SVM trained using truncated orthogonal polynomial kernels—kernels constructed so that the corresponding RKHS admits an explicit tensor-product orthonormal basis. Once training is complete, the fitted decision function coefficients (support vector weights and bias) are directly converted into coefficients in this basis using the orthonormality property, requiring no additional optimization. The OKC indices are then computed by summing the squared coefficients grouped by interaction order (main effects, pairwise, higher-order), polynomial degree, and individual coordinates, with normalization by the total squared RKHS norm. Finally, ORCA visualizations present how much of the classifier's magnitude comes from each type of interaction, enabling engineers to understand whether predictions rely on simple marginal effects or complex interactions.
Production Impact
For production systems using SVM classifiers, ORCA eliminates the need for post-hoc interpretability tools (like SHAP or LIME) that require separate model inference, approximations, or retraining. This reduces latency and complexity in model explanation pipelines—particularly valuable in regulated domains (finance, healthcare, insurance) where deterministic, auditable explanations are required. Engineers can immediately validate that high-stakes SVM models rely on reasonable feature interactions rather than spurious patterns, and can prune or simplify models by identifying redundant high-order interaction terms. However, the approach is restricted to SVM models with orthogonal polynomial kernels, so it cannot be applied to neural networks, other kernel types, or gradient boosting models that dominate modern production systems.
Limitations and When Not to Use This
This method only applies to SVMs with truncated orthogonal polynomial kernels, making it inapplicable to the neural networks, tree-based models, and other kernel types that dominate production deployments. The requirement for finite-dimensional RKHS with explicit orthonormal bases is fundamentally restrictive—classical RBF kernels, non-polynomial kernels, and infinite-dimensional spaces fall outside the framework. The paper does not address how to select truncation order or polynomial degree in advance, nor does it provide guidance on when ORCA explanations should be trusted for high-dimensional data where polynomial approximations may be poorly conditioned. Additionally, while the method is deterministic and parameter-free, it provides local explanations of the decision function structure rather than global model properties—following trends in modern interpretability, true causal understanding of what the model learned still requires additional domain expertise and validation.
Research Context
This work builds on decades of kernel method theory and the RKHS framework, extending classical results on orthogonal polynomials (Hermite, Legendre, Chebyshev) to SVM interpretability. It sits in the broader interpretability literature alongside SHAP, LIME, and attention-based explanations, but takes a fundamentally different route by leveraging mathematical structure rather than approximation or perturbation. The paper extends prior work on understanding SVM decision boundaries through kernel geometry and feature importance, now providing a structured decomposition that mirrors modern neural network interpretability techniques (e.g., interaction-order analysis in deep learning). This opens a research direction toward interpretable kernel machines that compete with black-box neural networks in regulated domains where explainability is non-negotiable.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::