Praxy Voice: Voice-Prompt Recovery + BUPS for Commercial-Class Indic TTS from a Frozen Non-Indic Base at Zero Commercial-Training-Data Cost
| Authors | Venkata Pushpak Teja Menta |
| Year | 2026 |
| HF Upvotes | 2 |
| arXiv | 2604.25441 |
| Download | |
| Code | https://github.com/praxelhq/praxy |
Abstract
Commercial TTS systems produce near-native Indic audio, but the best open-source bases (Chatterbox, Indic Parler-TTS, IndicF5) trail them on measured phonological dimensions, and the most widely adopted multilingual base (Chatterbox, 23 languages) does not even tokenise Telugu or Tamil. We ask: what is the minimum intervention that brings such a non-Indic-native base to commercial-class output on Telugu, Tamil, and Hindi, without training a new acoustic decoder and without any commercial TTS training data? We combine three pieces: (1) BUPS, a Brahmic Unified Phoneme Space that deterministically romanises seven Indic scripts to ISO-15919 so Chatterbox's Latin tokeniser can process them; (2) a LoRA adapter on only the text-token predictor (Chatterbox's t3), trained on ~1,220h of licensed Indic audio with a Hindi-proxy language_id; (3) a voice-prompt recovery recipe -- an 8-11s same-language reference clip plus three sampling overrides (exaggeration 0.7, temperature 0.6, min_p 0.1; "Config B") -- that recovers commercial-class acoustic output with no acoustic-decoder training. On Hindi, the LoRA regresses accuracy and we instead use vanilla Chatterbox + Config B, giving a two-branch deployment. Evaluated on 10-utterance pilot sets with the companion PSP benchmark, Praxy Voice matches or slightly leads commercial baselines: 26.7% retroflex collapse on Telugu (vs Sarvam Bulbul 33.3%), 71% Tamil-zha collapse (vs commercial trio's 86%), 0.025 LLM-WER on Hindi (tied with Cartesia Sonic-3). For intra-sentential code-mix we add a third branch (IndicF5 + native-script transliteration) that drops code-mix LLM-WER from 0.80-0.85 to 0.14-0.27 across Hi/Te/Ta. We release R6 LoRA weights (Apache-2.0), inference code and router (MIT), and a Gradio demo.
Engineering Breakdown
Plain English
This paper addresses a critical gap in open-source text-to-speech (TTS) for Indian languages. While commercial TTS systems produce near-native quality audio for Telugu, Tamil, and Hindi, the best open-source alternatives (Chatterbox, Indic Parler-TTS, IndicF5) significantly underperform on phonological quality, and the most widely-used multilingual base (Chatterbox with 23 languages) doesn't even tokenize Telugu or Tamil properly. The authors demonstrate that with minimal intervention—using a phoneme mapping scheme (BUPS) and a small LoRA adapter trained on ~1,220 hours of data—they can bring Chatterbox to commercial-class quality without retraining the acoustic decoder or using proprietary TTS data. This represents a practical, low-cost path to high-quality Indic language synthesis without building systems from scratch.
Core Technical Contribution
The paper's core innovation is BUPS (Brahmic Unified Phoneme Space), a deterministic romanization scheme that maps seven Indic scripts to ISO-15919 standard, enabling non-Indic-aware tokenizers like Chatterbox's Latin-based tokenizer to process Indic text correctly. Combined with this is a lightweight LoRA adapter applied only to the text-token predictor component (t3), which learns phoneme prediction without modifying the expensive acoustic decoder. The key insight is that the bottleneck for Indic language support in multilingual TTS is not the acoustic modeling (which works across languages) but the text processing pipeline—solving this at the token predictor level with LoRA allows transfer of a pre-trained decoder to new languages with minimal data and compute. This is fundamentally different from prior approaches that either retrain full models or require proprietary data.
How It Works
The system works in three stages. First, incoming Indic text in any of seven scripts (Devanagari, Telugu, Tamil, Kannada, Malayalam, Bengali, Gujarati) is passed through BUPS, which deterministically converts each script's characters to their phonetically-equivalent Latin representation following ISO-15919 romanization rules. This romanized text can now be tokenized by Chatterbox's existing Latin tokenizer, producing token sequences the model can consume. Second, these tokens enter the standard Chatterbox pipeline where the text-token predictor (t3) normally converts tokens to phone units; instead of using the base model's predictions, the LoRA adapter—a small set of learned weight matrices injected into t3—outputs refined phoneme predictions tuned to Indic phonology. Finally, the frozen acoustic decoder (which maps phonemes to spectrograms) generates mel-spectrograms, which are passed through existing vocoders to produce audio. The entire approach bypasses the need to retrain or modify the acoustic decoder, leveraging the fact that acoustic-to-audio mapping generalizes well across languages.
Production Impact
For teams building TTS systems for Indian languages, this approach dramatically reduces the engineering and data cost of supporting new languages. Instead of collecting or licensing 1,220+ hours of Indic-specific acoustic data and training a full model, you can adopt Chatterbox's proven acoustic decoder (already handling 20+ languages) and add language support with just a LoRA adapter and ~1,220 hours of text-audio data—a 5-10x reduction in compute and data requirements compared to full model training. The BUPS romanization scheme is simple enough to implement as a lightweight preprocessing step, adding minimal latency. In production, this means you can deploy high-quality TTS for Telugu, Tamil, and Hindi on smaller GPUs and with faster inference than alternatives, making it accessible to startups and smaller companies serving Indian language users. The trade-off is that you're still dependent on Chatterbox's acoustic quality as a ceiling—if you need significantly different voice characteristics or acoustic properties, you'll need a different approach.
Limitations and When Not to Use This
The paper relies on the assumption that Chatterbox's pre-trained acoustic decoder generalizes to Indic phonology, which may not hold for all edge cases or dialectal variation within these languages. BUPS is a deterministic romanization scheme that works for the seven tested Indic scripts but doesn't handle cross-script linguistic phenomena (like borrowed words from non-Indic languages within Indic text) in a linguistically sophisticated way. The approach requires ~1,220 hours of high-quality, aligned audio-text data for each language, which is a significant data requirement and may not exist for lower-resource Indic languages or regional dialects. The paper doesn't measure robustness to code-switching (mixing Indic and non-Indic languages in the same utterance), which is common in real-world Indian speech, so deployment would need additional validation on this failure mode. Finally, the method is specific to Chatterbox; while the BUPS concept could transfer to other TTS systems, the LoRA adapter approach requires architectures with adaptable text-token predictors, limiting its general applicability.
Research Context
This work builds on the broader trend of efficient fine-tuning in large models—specifically LoRA (Low-Rank Adaptation), which has proven effective for adapting pre-trained models to new domains with minimal parameter overhead. It addresses a documented gap identified in prior work comparing open-source Indic TTS systems (Indic Parler-TTS, IndicF5, Chatterbox), where tokenization and phonological handling were identified as bottlenecks rather than acoustic modeling. The paper implicitly leverages the finding from multilingual TTS research that acoustic decoders learned on diverse languages transfer well to new languages if phoneme predictions are accurate—this paper essentially operationalizes that insight by targeting the phoneme prediction layer specifically. It opens a research direction on minimal-intervention language adaptation: the question of what is the smallest architectural change and data investment needed to bring a multilingual model to commercial quality on a new language, which could apply beyond TTS to other sequence-to-sequence tasks (machine translation, speech synthesis, etc.).
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::