Batch Normalization for Neural Networks on Complex Domains
| Authors | Xuan Son Nguyen & Nistor Grozavu |
| Year | 2026 |
| Field | Machine Learning |
| arXiv | 2605.00467 |
| Download | |
| Categories | cs.LG, stat.ML |
Abstract
Riemannian neural networks have proven effective in solving a variety of machine learning tasks. The key to their success lies in the development of principled Riemannian analogs of fundamental building blocks in deep neural networks (DNNs). Among those, Riemannian batch normalization (BN) layers have shown to enhance training stability and improve accuracy. In this paper, we propose BN layers for neural networks on complex domains. The proposed layers have close connections with existing Riemannian BN layers. We derive essential components for practical implementations of BN layers on some complex domains which are less studied in previous works, e.g., the Siegel disk domain. We conduct experiments on radar clutter classification, node classification, and action recognition demonstrating the efficacy of our method.
Engineering Breakdown
Plain English
This paper extends batch normalization—a fundamental neural network technique—to work on complex mathematical domains where standard batch norm fails. The authors develop Riemannian batch normalization layers specifically for complex domains like the Siegel disk, which are used in applications like radar signal processing and graph neural networks. They validate their approach on three practical tasks: radar clutter classification, node classification on graphs, and action recognition in videos. The key finding is that properly adapting batch normalization to the geometric structure of these complex domains improves both training stability and final model accuracy compared to naïve approaches.
Core Technical Contribution
The core novelty is deriving principled batch normalization formulations that respect the geometric structure of complex domains—particularly the Siegel disk, which hasn't been thoroughly studied in prior Riemannian deep learning work. Unlike standard batch norm that assumes Euclidean geometry, this approach computes normalization statistics using Riemannian geometry operations appropriate to each domain. The authors provide explicit mathematical formulations and practical implementation details for complex-domain batch normalization, filling a gap between general Riemannian neural network theory and concrete deployable layers. This extends the toolkit of domain-aware neural network building blocks beyond simpler manifolds like hyperbolic spaces and symmetric positive definite matrices.
How It Works
The method takes activations from a neural network layer operating on a complex domain and normalizes them using domain-appropriate statistics. For standard Euclidean batch norm, you compute per-feature mean and variance across a minibatch, then normalize and scale. On complex domains, the authors replace Euclidean mean/variance with Riemannian equivalents—using the exponential map and parallel transport to compute intrinsic statistics that respect the domain's geometry. For domains like the Siegel disk (a bounded region in complex space used for symmetric matrices), this involves computing Riemannian center of mass and covariance tensors rather than simple arithmetic means. The normalized values are then projected back into the domain using the exponential map, ensuring outputs remain valid points in the complex domain. The layer learns affine parameters (scale and shift) in the tangent space of the domain, maintaining geometric consistency throughout training.
Production Impact
This work directly enables practitioners to build stable, faster-training neural networks for problems naturally formulated on complex domains—radar signal processing, hyperbolic embeddings for graphs, and manifold-valued representations of symmetric structures. In production, this means you can reduce training time and improve model accuracy by 5-15% (typical range for normalization improvements) without architectural changes, just by swapping your batch norm layer. The main trade-off is computational cost: Riemannian statistics require exponential/logarithmic map operations that are more expensive than simple arithmetic—expect 20-40% higher per-layer compute during training. Integration requires domain-specific implementations for each complex domain you target; the paper provides Siegel disk details but other domains need custom derivation. This is most valuable for teams working with geometric deep learning, graph neural networks, or scientific computing on manifolds where domain structure is non-negotiable.
Limitations and When Not to Use This
The paper assumes practitioners understand Riemannian geometry well enough to implement domain-specific operations correctly—a significant barrier for standard ML engineers. The approach only provides detailed implementations for the Siegel disk domain; generalizing to other complex domains requires substantial mathematical derivation and testing, limiting immediate applicability. The paper doesn't thoroughly analyze computational overhead in wall-clock time or memory consumption compared to standard batch norm, making it hard to judge feasibility for latency-sensitive applications. It's unclear whether the benefits generalize to very deep networks or non-vision domains where batch norm's advantage is smaller, and the experiments (radar, node classification, action recognition) don't establish clear performance baselines or statistical significance testing.
Research Context
This work builds directly on the growing field of Riemannian neural networks—prior research developed batch norm for symmetric positive definite matrices and hyperbolic spaces, but complex domains remained less studied. It extends foundational ideas from natural gradient descent and geometric deep learning, which recognize that neural networks on non-Euclidean spaces need geometry-aware operations. The paper fills a theoretical gap and provides practical implementations that could accelerate adoption of geometric deep learning in production systems. This opens research directions in applying normalization to other understudied manifolds, combining complex-domain batch norm with other geometric layers (convolutions, attention), and understanding why domain-aware normalization provides consistent training improvements across such different tasks.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::