TLDR: A new method, CODEADAPT, combines code execution with minimal in-context learning to enable standard language models (LMs) to match or exceed the reasoning performance of more expensive, reinforcement learning-trained reasoning models (RMs). It achieves this across diverse tasks, including instruction following, language processing, and formal reasoning, while also being more token-efficient and cheaper to train and run. The approach allows LMs to dynamically blend natural language and code for problem-solving and exhibits metacognitive behaviors.
A recent research paper titled “CODE-ENABLED LANGUAGE MODELS CAN OUTPERFORM REASONING MODELS ON DIVERSE TASKS” by Cedegao E. Zhang, Cédric Colas, Gabriel Poesia, Joshua B. Tenenbaum, and Jacob Andreas introduces a novel approach called CODEADAPT. This method demonstrates that standard language models (LMs), when equipped with code execution capabilities and minimal in-context learning, can achieve reasoning performance comparable to, or even surpassing, more expensive and complex reasoning models (RMs).
Reasoning models, which are LMs trained with reinforcement learning to generate extensive natural language reasoning, have shown impressive results. However, their development and deployment come with significant costs in terms of computational resources and data. The authors of this paper questioned whether advanced reasoning necessarily requires such substantial investment, proposing a more economical alternative.
CODEADAPT combines two key techniques: the CodeAct framework and Generalization-guided Few-shot Learning (GFL). The CodeAct framework allows language models to seamlessly integrate natural language reasoning with code execution in a multi-step process. This means the LM can think in natural language, then write and execute Python code to perform calculations, verify hypotheses, or break down problems, and then interpret the results to continue its reasoning. This creates a dynamic workspace where the model can offload structured computations, much like a human might use a calculator or scratchpad.
The second component, GFL, is a lightweight learning procedure that uses as few as five training problems per task domain. Unlike traditional methods that might select examples based solely on correctness, GFL identifies solutions that are most “instructive” – those that help the model generalize and solve *other* problems effectively. This self-generated learning experience allows models to discover robust problem-solving strategies without requiring extensive fine-tuning or reinforcement learning.
The researchers conducted a systematic evaluation, comparing four popular instruct LMs (DeepSeek V3, Gemini 2.0 Flash, Qwen 3 30B A3B Instruct, and Qwen 2.5 Coder 32B) against their corresponding reasoning-trained counterparts. These models were tested across eight diverse tasks, including instruction following, language processing, and formal reasoning (like mathematical problems and logic puzzles).
The results were compelling. CODEADAPT enabled three out of the four LMs to outperform their corresponding RMs on average across eight tasks, with performance gains up to 22.9% for individual models and 35.7% across tasks. Furthermore, CodeAdapt proved to be significantly more token efficient, using 10-81% fewer tokens during inference. This translates to lower operational costs and faster execution.
An interesting finding was that while reasoning models showed significant improvements in formal reasoning and instruction following, their impact on linguistic reasoning tasks was mixed, sometimes even leading to performance degradation. In contrast, CodeAdapt consistently improved performance across all domains, including language processing tasks. The study also highlighted that CodeAdapt’s performance benefits from allowing more iterative reasoning turns, indicating that providing more opportunities for interaction and refinement leads to better outcomes.
The analysis of CodeAdapt’s problem-solving behavior revealed that it dynamically adapts its strategies to the specific demands of each task. For instance, logical constraint problems saw a predominantly code-based approach, while creative ideation relied more on natural language. The system also exhibited metacognitive abilities, such as monitoring solution progress, recognizing its own limitations, and reasoning about resource constraints, often leading to strategy shifts when errors were encountered or resources became scarce.
Also Read:
- Enhancing Mathematical Reasoning in Language Models: A Reinforcement Learning Approach to Budget Forcing
- Bridging the Gap: How Symbolic AI Enhances Transparency and Reasoning in Large Language Models
This research suggests that powerful reasoning capabilities might already be inherent in modern instruct-tuned LMs, and frameworks like CodeAdapt provide an effective and cost-efficient way to unlock them. It also underscores the potential of hybrid reasoning systems that leverage the complementary strengths of neural language processing and symbolic computation. For more technical details, you can refer to the full paper here.
