Building Multilingual Bridges: Data Mixing Unlocks In-Language Reasoning in Tiny Aya
Original title:Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoning
Most reasoning models default to English internal monologues regardless of the input language, often flattening cultural nuance and user intent. Researchers address this with Tiny Aya L2-Thinker, a 3.35B parameter model designed for consistent in-language (L2) reasoning. By optimizing data mixture and scheduling during supervised fine-tuning, the model maintains an L2 reasoning rate above 93% across 60 languages on six benchmarks. The findings show that reasoning behavior is fundamentally language-agnostic: a strong English reasoning core combined with broad multilingual non-reasoning data suffices to generalize reasoning to unseen languages without per-language chain-of-thought supervision.
Why it's worth reading
It offers an efficient, data-centric blueprint to dismantle English-centric reasoning biases, demonstrating that native-language chain-of-thought can transfer to unseen languages without expensive per-language reasoning supervision.