Neural Operators Can Discover Functional Clusters
| Authors | Yicen Li et al. |
| Year | 2026 |
| Field | Machine Learning |
| arXiv | 2602.23528 |
| Download | |
| Categories | cs.LG, cs.CE, stat.CO, stat.ML |
Abstract
Operator learning is reshaping scientific computing by amortizing inference across infinite families of problems. While neural operators (NOs) are increasingly well understood for regression, far less is known for classification and its unsupervised analogue: clustering. We prove that sample-based neural operators can learn any finite collection of classes in an infinite-dimensional reproducing kernel Hilbert space, even when the classes are neither convex nor connected, under mild kernel sampling assumptions. Our universal clustering theorem shows that any closed classes can be approximated to arbitrary precision by NO-parameterized classes in the upper Kuratowski topology on closed sets, a notion that can be interpreted as disallowing false-positive misclassifications. Building on this, we develop an NO-powered clustering pipeline for functional data and apply it to unlabeled families of ordinary differential equation (ODE) trajectories. Discretized trajectories are lifted by a fixed pre-trained encoder into a continuous feature map and mapped to soft assignments by a lightweight trainable head. Experiments on diverse synthetic ODE benchmarks show that the resulting practical SNO recovers latent dynamical structure in regimes where classical methods fail, providing evidence consistent with our universal clustering theory.
Engineering Breakdown
Plain English
This paper extends neural operators—a class of models that learn mappings between infinite-dimensional function spaces—to handle unsupervised clustering and classification tasks. The authors prove a universal approximation theorem showing that neural operators can discover any finite collection of K classes in reproducing kernel Hilbert space, even when classes are non-convex and disconnected. The key insight is that sample-based neural operators can learn to partition infinite families of problems while respecting a topology that prevents false-positive misclassifications. This bridges operator learning (previously understood mainly for regression) into the clustering domain with formal guarantees.
Core Technical Contribution
The paper's core novelty is a universal approximation theorem for neural operators in clustering tasks, extending beyond the well-studied regression case. The authors introduce a formal framework using the upper Kuratowski topology on closed sets to guarantee that learned class boundaries avoid false positives—a stronger guarantee than traditional classification metrics. They prove that any K disjoint closed classes can be approximated to arbitrary precision by NO-parameterized classes under mild kernel sampling assumptions, without requiring convexity or connectedness. This is the first theoretical result connecting neural operator expressivity to unsupervised clustering in infinite dimensions.
How It Works
Neural operators learn mappings from one function space to another by parameterizing operators as compositions of linear and nonlinear transformations in spectral or spatial domains. In the clustering case, the NO takes a sample from a function (representing an infinite-dimensional object) as input and outputs class membership or cluster assignments. The proof relies on the fact that the reproducing kernel Hilbert space (RKHS) equipped with the upper Kuratowski topology provides a natural metric on closed sets, allowing arbitrary-precision approximation. The algorithm constructs NO-parameterized decision boundaries by learning kernel representations that implicitly partition the function space, with the topology ensuring that the learned boundaries remain closed and properly separated. During inference, given a new function sample, the operator evaluates it through learned kernel-based features to determine cluster membership. The key technical mechanism is that operator linearity (after nonlinear lifting) preserves the topological structure needed to avoid false positives.
Production Impact
For engineers building scientific computing or data analysis pipelines, this work enables amortized clustering inference: instead of solving separate clustering problems for each new dataset, a single trained neural operator handles infinite families of related problems, dramatically reducing per-instance inference cost. In fields like material science, climate modeling, or medical imaging where you work with function-valued data (e.g., time series, spatial fields), this approach eliminates the need to handcraft features or run separate clustering algorithms for each sample. However, production adoption requires: (1) careful kernel selection and sampling strategy—the theory assumes 'mild' assumptions that need validation per domain, (2) GPU memory for learning high-dimensional spectral representations, and (3) validation that the Kuratowski topology's false-positive guarantee aligns with your actual cost function (since the theorem optimizes for topology, not accuracy metrics). The latency benefit is significant—inference on new functions is O(d) with learned features rather than O(n²) distance-based clustering—but only if your training data captures the true distribution of test functions.
Limitations and When Not to Use This
The paper assumes mild kernel sampling conditions that are not fully specified and may not hold for irregularly-sampled or high-noise function data in practice. The upper Kuratowski topology prevents false positives but provides no guarantees on false negatives or clustering quality metrics (purity, NMI), which matter more in some applications. The proof requires K classes to be closed in RKHS, excluding cases with fuzzy or overlapping cluster boundaries that exist in real data. The paper doesn't address computational complexity of training the neural operator—how many samples are needed, what is the sample complexity with respect to function space dimension and K? It also lacks empirical validation: no experiments on real or synthetic datasets compare this approach to classical clustering, so practical effectiveness remains unknown. The framework assumes access to infinite function samples but provides no guidance on how many finite samples suffice.
Research Context
This work builds on two decades of neural operator research (DeepONet, Fourier Neural Operators) that have proven effective for learning PDE solvers and regression tasks, but extends the theory to unsupervised learning for the first time. It fills a gap in operator learning literature where classification and clustering were largely unexplored compared to dense regression. The paper connects to classical functional data analysis and RKHS theory, bringing modern deep learning guarantees into that more mature mathematical framework. Future work will likely focus on empirical validation, sample complexity bounds, and extensions to semi-supervised or active learning settings where partial labels are available.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::