TLDR: Large Reasoning Models (LRMs) generally perform better when reasoning in English, even for non-English questions, exhibiting higher accuracy and more cognitive behaviors. However, this English-centric strategy is susceptible to “Lost in Translation” errors, where translation mistakes lead to incorrect answers that would have been avoided by reasoning in the question’s original language, particularly in low-resource languages. The research highlights the critical need to develop robust native-language reasoning capabilities in AI.
Large Reasoning Models (LRMs) have become increasingly popular for their ability to analyze and answer complex questions across various tasks, from mathematics to science. Unlike traditional Large Language Models (LLMs), LRMs employ a two-phase response generation: first, they generate a step-by-step reasoning sequence, and then they produce a succinct final answer. While powerful, their multilingual reasoning capabilities have remained largely unexplored.
A recent study, titled The Reasoning Lingua Franca: A Double-Edged Sword for Multilingual AI, delves into this critical area. Authored by Alan Saji, Raj Dabre, Anoop Kunchukuttan, and Ratish Puduppully, the research systematically compares an LRM’s reasoning performance when operating in English versus when reasoning in the language of the original question.
The English Advantage in Reasoning
The study evaluated LRMs on two benchmark datasets, MGSM and GPQA Diamond, which vary in difficulty. A significant finding was that LRMs often default to reasoning in English, even when presented with non-English questions. This English-centric approach generally yielded higher final-answer accuracy. The performance gap between English reasoning and reasoning in the question’s native language was observed to widen as tasks became more complex and for lower-resource languages. For instance, on the GPQA Diamond task, which requires expert-level domain knowledge, the contrast between English and non-English reasoning was particularly stark.
Beyond just accuracy, the researchers also analyzed cognitive attributes within the reasoning traces, such as sub-goal setting, verification, backtracking, and backward chaining. These behaviors, which reflect how human experts tackle difficult problems, were found to be substantially more present when the models reasoned in English. This suggests that English reasoning allows LRMs to exhibit richer, more sophisticated cognitive processes.
The “Lost in Translation” Vulnerability
Despite the general advantage of English reasoning, the study uncovered a critical failure mode termed “Lost in Translation.” This occurs when translation steps introduce errors into the English reasoning process, leading to incorrect answers that would have been avoided if the model had reasoned directly in the question’s original language. These translation-induced errors were found to be more common in low-resource languages, with the fraction of incorrect answers due to mistranslation decreasing as languages became more high-resource.
For example, in one instance, an English reasoning trace misinterpreted a key detail in a Hindi question, leading to an incorrect final answer. The original Hindi reasoning, without the translation step, correctly arrived at the solution. This highlights a systemic weakness: while English reasoning might leverage an LRM’s domain knowledge more effectively, it risks being undermined by inaccuracies introduced during the translation of multilingual inputs.
Also Read:
- Calibrating Large Reasoning Models for Smarter Machine Translation Evaluation
- Unmasking LLM Reflection: Why Self-Correction Falls Short in Open-Ended Tasks
Implications for Multilingual AI Development
The findings underscore a crucial dilemma for multilingual AI. While reasoning in English often leads to better performance and more evident cognitive behaviors, it introduces a vulnerability to translation errors. This suggests that simply translating non-English inputs into English for reasoning is not sufficient for robust multilingual AI. The researchers emphasize the need for dedicated efforts in dataset construction, training objectives, and evaluation methods that specifically aim to develop and benchmark native-language reasoning capabilities with the same rigor currently applied to English.
The study provides an empirical diagnosis of this problem and establishes baselines for future work, paving the way for more reliable and culturally nuanced multilingual reasoning in AI models.
