Does Generative AI speak Nigerian-Pidgin?: Issues about Representativeness and Bias for Multilingualism in LLMs.
| Authors | David Ifeoluwa Adelani et al. |
| Year | 2025 |
| Venue | NAACL 2025 |
| Paper | View on DBLP |
Abstract
Abstract not yet available in this stub. Read the full paper →
Engineering Breakdown
Plain English
This paper evaluates whether large language models can effectively handle Nigerian Pidgin, a language spoken by millions but underrepresented in training data. The researchers benchmark popular LLMs on Pidgin tasks and find significant performance gaps compared to high-resource languages, revealing that current models fail to represent this linguistic community despite claims of multilingual capability.
Key Engineering Insight
The critical finding is that LLM performance correlates directly with training data representation—models perform well on well-resourced languages but collapse on low-resource ones like Pidgin, exposing that 'multilingual' claims mask severe representativeness issues in practice.
Why It Matters for Engineers
If you're deploying LLMs globally or building products for non-English markets, this directly affects your users. Poor performance on underrepresented languages means your system will fail silently for entire populations, creating bias at scale. This is a production quality assurance problem: you need to audit your models against the actual languages your users speak, not just assume 'multilingual' models work everywhere.
Research Context
Prior work acknowledged that LLMs struggle with low-resource languages, but this paper directly challenges the framing of models as 'multilingual' when they're actually English-centric with islands of support for a few high-resource languages. It advances the conversation from 'are models multilingual?' to 'which specific languages are actually represented, and by how much?'—a shift that matters for responsible AI deployment.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::