Convergent Evolution: How Different Language Models Learn Similar Number Representations
| Authors | Deqing Fu et al. |
| Year | 2026 |
| HF Upvotes | 6 |
| arXiv | 2604.20817 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Language models trained on natural text learn to represent numbers using periodic features with dominant periods at T=2, 5, 10. In this paper, we identify a two-tiered hierarchy of these features: while Transformers, Linear RNNs, LSTMs, and classical word embeddings trained in different ways all learn features that have period-T spikes in the Fourier domain, only some learn geometrically separable features that can be used to linearly classify a number mod-T. To explain this incongruity, we prove that Fourier domain sparsity is necessary but not sufficient for mod-T geometric separability. Empirically, we investigate when model training yields geometrically separable features, finding that the data, architecture, optimizer, and tokenizer all play key roles. In particular, we identify two different routes through which models can acquire geometrically separable features: they can learn them from complementary co-occurrence signals in general language data, including text-number co-occurrence and cross-number interaction, or from multi-token (but not single-token) addition problems. Overall, our results highlight the phenomenon of convergent evolution in feature learning: A diverse range of models learn similar features from different training signals.
Engineering Breakdown
Plain English
This paper investigates how language models learn to represent numbers through periodic features in the Fourier domain, discovering that models trained on natural text exhibit spikes at periods T=2, 5, and 10. The authors identify a critical gap: while many architectures (Transformers, LSTMs, Linear RNNs, word embeddings) learn these periodic features, only some learn features that are geometrically separable enough to linearly classify numbers modulo T. Through theoretical analysis, they prove that Fourier domain sparsity is necessary but insufficient for achieving mod-T geometric separability, then empirically demonstrate that data, architecture, optimizer, and tokenizer choices all significantly influence whether models learn geometrically separable representations.
Core Technical Contribution
The paper's core novelty is establishing a formal hierarchy distinguishing between two levels of numeric feature learning: (1) basic Fourier-domain periodicity that many models naturally learn, and (2) geometric separability that enables linear classification mod-T, which requires specific training conditions. The authors prove this distinction rigorously, showing that sparsity in frequency domain is necessary but not sufficient for the stronger geometric property. They then empirically characterize which combinations of data, architecture, optimizer, and tokenizer design lead to geometrically separable features, providing actionable insights into what training decisions unlock linear separability. This bridges a gap between what we observe in model internals (periodic patterns) and what we can actually use for downstream tasks (linear classifiers).
How It Works
The investigation begins by taking trained language models and analyzing how they encode numerical values by examining their activation patterns in Fourier space, looking for characteristic spikes at specific periods. The authors measure both the presence of periodic features (which many models show) and the geometric separability of those features—whether a linear decision boundary can cleanly separate different residue classes modulo T. They conduct a controlled experiment varying training conditions: different tokenizers, optimizers, architectures, and datasets, then measure the resulting feature separability. Theoretical analysis uses Fourier domain properties to characterize necessity conditions, while empirical evaluation uses linear probing and geometric measurements to assess separability. The key technical insight is that periodic patterns visible in frequency space don't automatically translate to features arranged in geometrically distinct clusters in embedding space, requiring additional analysis to understand which training regimes produce the stronger property.
Production Impact
For production systems, this research directly impacts any application requiring numerical reasoning in language models—financial calculations, scientific computing, structured data processing, and quantity estimation. Understanding that basic periodicity doesn't guarantee usable numeric representations means engineers need to be deliberate about training choices: model architecture, tokenizer design, and optimizer selection all measurably affect whether models can accurately manipulate numbers. If you're fine-tuning a model for numeric tasks, you can't simply rely on hoping the base model learned good numeric representations; you need to verify geometric separability of numeric features, potentially requiring targeted training with specific optimizers or data augmentation strategies. The trade-off is that ensuring geometrically separable numeric features may require additional training compute and careful hyperparameter tuning, but the payoff is reliable numeric reasoning. This suggests production pipelines should include probing tasks that verify numeric feature quality before deploying models on math-heavy applications.
Limitations and When Not to Use This
The paper focuses on relatively simple modular arithmetic (mod-T classification) rather than complex numerical reasoning like multi-step arithmetic or scaling to very large numbers, so insights may not directly transfer to harder numeric tasks. The identified periods T=2, 5, 10 emerge from specific natural language datasets; different corpora or languages might exhibit different periodicities, and the paper doesn't fully explore generalization across diverse data regimes. The empirical findings about which combinations of architecture, optimizer, tokenizer, and data yield separability are specific to the tested configurations; there's a combinatorial explosion of possible choices not covered, limiting prescriptive guidance. Additionally, the paper doesn't address whether geometrically separable features in training actually predict downstream numeric task performance on held-out applications, leaving open whether this property is sufficient for practical numeric reasoning in production systems.
Research Context
This work builds on recent interest in mechanistic interpretability of language models, particularly how they represent and manipulate abstract concepts like numbers and mathematics. It extends prior research on periodic structure in neural networks and Fourier analysis of learned features, grounding these observations in both theoretical necessity conditions and empirical characterization. The paper advances the growing field of understanding numeric capabilities in language models, joining concurrent work investigating why LLMs struggle with arithmetic and how architectural choices affect numerical reasoning. By establishing a two-tier framework (periodicity vs. geometric separability), it provides vocabulary and tools for future work investigating other abstract conceptual representations in neural networks and informs the design of models intended for mathematical or structured reasoning tasks.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::