Optimizing Korean-Centric LLMs via Token Pruning
| Authors | Hoyeol Kim & Hyeonwoo Kim |
| Year | 2026 |
| Field | NLP |
| arXiv | 2604.16235 |
| Download | |
| Categories | cs.CL |
Abstract
This paper presents a systematic benchmark of state-of-the-art multilingual large language models (LLMs) adapted via token pruning - a compression technique that eliminates tokens and embedding parameters corresponding to languages irrelevant to the target application. Focusing on Korean-centric natural language processing (NLP) tasks, we evaluate architectures including Qwen3, Gemma-3, Llama-3, and Aya across three vocabulary configurations: Original, English-Korean (EnKo), and English-Korean-Chinese (EnKoZh). Performance is assessed using established benchmarks for general aptitude, cultural literacy, instruction following, and machine translation. Our findings indicate that token pruning significantly improves generation stability by eliminating language confusion, and in the case of machine translation, frequently enhances performance on Korean-specific tasks. While instruction-following capabilities display architecture-dependent variance linked to latent cross-lingual representations, the significant reduction in vocabulary size validates token pruning as a highly effective optimization strategy for memory-constrained, domain-specific deployments, despite modest gains in inference latency.
Engineering Breakdown
Plain English
This paper evaluates token pruning—a compression technique that removes vocabulary and embedding parameters for irrelevant languages—on Korean-focused LLM tasks. The authors test four state-of-the-art models (Qwen3, Gemma-3, Llama-3, Aya) across three vocabulary setups: the full original vocabulary, an English-Korean subset, and an English-Korean-Chinese subset. They measure performance on general aptitude, cultural literacy, instruction following, and machine translation benchmarks. The key finding is that token pruning significantly improves generation stability by preventing language confusion, suggesting that eliminating cross-lingual noise helps models focus on target-language performance.
Core Technical Contribution
The paper's core contribution is demonstrating that selective vocabulary pruning—removing embeddings and tokens for irrelevant languages—is a practical compression strategy that actually improves downstream performance on language-specific tasks rather than degrading it. Unlike prior work on general model compression (quantization, distillation), token pruning is linguistically-targeted: it exploits the fact that multilingual models waste parameter capacity on languages they'll never use. The systematic benchmark across multiple model architectures and vocabulary configurations provides empirical evidence that this approach generalizes, and the finding that it reduces 'language confusion' suggests a fundamental benefit beyond pure compression.
How It Works
Token pruning operates by first identifying which languages are relevant to the target application (in this case, Korean). The technique then removes embedding rows and corresponding token IDs from the vocabulary that represent irrelevant languages, effectively shrinking the embedding matrix and reducing the model's parameter count. During inference, the model's input and output layers now map only to the pruned vocabulary, meaning tokens for Chinese, Spanish, or other irrelevant languages cannot be generated or processed. The authors evaluate three configurations: keeping the full vocabulary (baseline), pruning to English+Korean only, and pruning to English+Korean+Chinese, allowing comparison of how aggressive pruning affects performance. The compressed models are not retrained end-to-end; instead, the pruning is applied to pre-trained weights, making this a post-training optimization technique.
Production Impact
For teams deploying Korean-centric LLMs in production, token pruning offers immediate benefits: reduced model size (smaller embedding matrix means fewer parameters to load and fewer GPU memory requirements), faster token generation (smaller vocabulary means smaller logits computations), and improved inference stability by eliminating cross-lingual interference. A production pipeline could adopt this by: (1) measuring which languages actually appear in your deployment domain, (2) pruning the vocabulary post-training (no expensive retraining required), (3) reducing serving costs via smaller model footprints and faster token generation. The trade-off is loss of zero-shot capability for non-target languages—if your system unexpectedly needs to handle Chinese or Japanese input, a pruned model will fail. For specialized deployments (e.g., Korean customer support, Korean content generation), this is acceptable; for general-purpose systems, the risk is real. The technique also requires identifying the actual vocabulary cutoff point empirically per use case.
Limitations and When Not to Use This
The paper's scope is limited to Korean-centric applications, and generalization to other language pairs or polyglot systems remains unclear—pruning to EnKoZh may have different trade-offs than pruning to, say, English-French-German. The abstract suggests results are incomplete ('in the case of ma...' cuts off), so we cannot evaluate all findings or potential failure modes. Token pruning assumes your target languages are known and fixed at deployment time; dynamic, multilingual applications cannot benefit. The paper doesn't address what happens when pruned models encounter code-switching (mixing Korean and English in the same prompt), which is common in real-world use—the mechanism for handling such input is unclear. Additionally, no analysis is provided on whether performance gains stem from compression itself or from preventing specific language interference patterns, which would be crucial for extending this to other language families.
Research Context
This work builds on the broader trend of parameter-efficient fine-tuning and model compression (LoRA, quantization, pruning), but applies a linguistic lens: instead of pruning by weight magnitude or layer, it prunes by language relevance. It relates to prior work on vocabulary compression for low-resource deployment and multilingual model analysis, which has shown that multilingual models often suffer from interference between languages. The paper contributes to the growing research area of language-specific model optimization, responding to the reality that not all deployed models need all languages. It opens a research direction: if removing irrelevant languages improves performance, can we design multilingual models that avoid this interference entirely, or is selective pruning a practical solution we should standardize for production deployments?
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::