You Can't Fight in Here! This is BBS!
| Authors | Richard Futrell & Kyle Mahowald |
| Year | 2026 |
| Field | NLP |
| arXiv | 2604.09501 |
| Download | |
| Categories | cs.CL |
Abstract
Norm, the formal theoretical linguist, and Claudette, the computational language scientist, have a lovely time discussing whether modern language models can inform important questions in the language sciences. Just as they are about to part ways until they meet again, 25 of their closest friends show up -- from linguistics, neuroscience, cognitive science, psychology, philosophy, and computer science. We use this discussion to highlight what we see as some common underlying issues: the String Statistics Strawman (the mistaken idea that LMs can't be linguistically competent or interesting because they, like their Markov model predecessors, are statistical models that learn from strings) and the As Good As it Gets Assumption (the idea that LM research as it stands in 2026 is the limit of what it can tell us about linguistics). We clarify the role of LM-based work for scientific insights into human language and advocate for a more expansive research program for the language sciences in the AI age, one that takes on the commentators' concerns in order to produce a better and more robust science of both human language and of LMs.
Engineering Breakdown
Plain English
This paper is a meta-discussion about whether modern large language models can meaningfully contribute to linguistics and cognitive science research, presented as a dialogue between two researchers and 25 colleagues from related fields. Rather than presenting novel experimental results, the authors identify and critique two pervasive misconceptions: the 'String Statistics Strawman' (the false belief that statistical models learning from text can't be linguistically competent) and the 'As Good As it Gets Assumption' (that current LM research in 2026 represents the ceiling of what's possible). The paper uses this framework to structure a wide-ranging discussion about the theoretical foundations of language models and their implications for understanding human language.
Core Technical Contribution
The core contribution is a conceptual clarification rather than a technical innovation—the authors systematically deconstruct two widespread but flawed arguments that dominate the linguistics-versus-AI debate. They argue that dismissing LMs as 'mere string statistics' conflates the learning mechanism with linguistic competence, and that prematurely concluding LM research has plateaued prevents serious engagement with fundamental questions about language. By framing these as named fallacies and assembling diverse expert perspectives, they create a structured argument for why linguistics and AI should have a more productive dialogue, legitimizing LMs as objects of serious linguistic inquiry rather than dismissing them on principled grounds.
How It Works
The paper employs a Socratic dialogue structure, beginning with a conversation between a formal linguist (Norm) and a computational scientist (Claudette), then expanding to include 25 additional expert voices from linguistics, neuroscience, cognitive science, psychology, philosophy, and computer science. Rather than a traditional empirical study, each section addresses a specific criticism or misconception about LMs by presenting counterarguments, relevant evidence, and theoretical frameworks from multiple disciplines. The progression moves from diagnosing the core fallacies (String Statistics Strawman and As Good As it Gets Assumption) through examining what LMs actually learn, what they reveal about language structure, and how they relate to human cognition. The output is a comprehensive reframing of the LM-linguistics relationship that positions these models as valuable research tools for understanding language rather than threats to or competitors with traditional linguistics.
Production Impact
For engineers building production NLP systems, this paper's main value is conceptual clarity about what language models actually are and what they're good for, which directly affects how you structure research roadmaps and feature development. The paper argues against the defeatist view that current LM architectures can't improve—if practitioners internalize this, they're more likely to invest in understanding failure modes and scaling limitations rather than assuming we've hit a ceiling. In practice, this means taking linguistic phenomena seriously when designing evaluation benchmarks and error analysis, rather than treating LM improvements as purely engineering problems. The multi-disciplinary framing also suggests that production systems should incorporate insights from cognitive science and linguistics (e.g., compositionality, recursion handling, agreement constraints) into both training objectives and post-hoc validation, though this adds engineering complexity and validation overhead.
Limitations and When Not to Use This
This is fundamentally a position paper and meta-analysis rather than an empirical study, so it provides no new experimental results, benchmarks, or datasets that engineers can directly apply to improve their systems. The paper's arguments are theoretical and philosophical—while they make conceptual progress on what LMs are, they don't directly address practical problems like how to make models faster, more efficient, or more aligned with human values. The 'As Good As it Gets Assumption' critique assumes continued progress is possible but doesn't specify what architectural changes or training approaches would actually overcome current limitations—the paper opens questions rather than closing them. Additionally, the Socratic format may obscure sharp technical disagreements; real production constraints (latency, cost, data availability) that force trade-offs between linguistic sophistication and practical deployability are not deeply explored.
Research Context
This paper sits at the intersection of formal linguistics, cognitive science, and modern NLP, building on decades of debate about whether statistical methods can capture linguistic structure (going back to Markov models and n-gram research). It directly engages with the 'neural network skepticism' that arose after the success of Transformers, which forced a reckoning between the empirical success of LMs and theoretical arguments that string statistics shouldn't work for language. The paper echoes and extends critiques of the String Statistics Strawman from work by researchers like Yonatan Belinkov and others studying what linguistic structure emerges in LLMs, while challenging the implicit pessimism in some linguistics circles about whether neural models can be scientifically interesting. It opens a research direction toward using LMs as instruments for testing linguistic theories (similar to how Norvig and others have advocated), rather than only viewing them as engineering artifacts.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::