TLDR: ILR (InteractiveLearning for LLM Reasoning) is a new framework that enables Large Language Models (LLMs) to enhance their independent problem-solving abilities through multi-agent interaction. It uses “Dynamic Interaction” to adaptively choose cooperation or competition based on problem difficulty, employing a three-stage “Idea3” process (Sharing, Analysis, Fusion). “Perception Calibration” provides automated, fine-grained reward signals for learning. Experiments show ILR consistently improves LLM performance on reasoning and coding benchmarks, demonstrating that interactive learning can make individual LLMs more robust and capable than traditional self-learning.
Large Language Models (LLMs) have shown incredible potential, but enhancing their reasoning abilities remains a key challenge. Traditionally, LLMs learn through methods like Supervised Fine-Tuning or Reinforcement Learning, which can be thought of as a form of self-learning. However, human learning often thrives in interactive environments, where individuals collaborate, challenge each other, and provide feedback to deepen understanding and find more robust solutions. This concept of multi-agent learning, where multiple LLMs interact, is now being explored to see if it can similarly boost their problem-solving skills.
Existing multi-agent learning approaches for LLMs have focused on creating collaborative environments to build stronger multi-agent systems. While these systems perform well, they typically require all agents to re-execute their processes during inference to arrive at a final solution. This differs from how humans learn; individuals improve through interaction and then can solve problems independently in the future. A new framework, called ILR (InteractiveLearning for LLM Reasoning), aims to bridge this gap by investigating whether multi-agent interaction can enhance an LLM’s independent problem-solving ability.
Introducing ILR: A Co-Learning Framework
ILR is a novel co-learning framework for multi-agent systems that integrates two crucial components: Dynamic Interaction and Perception Calibration. These components are designed to mimic human discussion and learning processes, allowing LLMs to improve their individual reasoning capabilities.
Dynamic Interaction: Mimicking Human Discussion
The Dynamic Interaction module is inspired by how humans adapt their strategies based on problem complexity. For difficult problems, people tend to cooperate, while for simpler ones, they might compete to find the most efficient solution. ILR emulates this by having an LLM estimate a question’s difficulty through self-ranking and then use Item Response Theory to calculate the probability of solving it independently. If the probability is low, the model engages in cooperation; otherwise, it chooses competition.
At the heart of Dynamic Interaction is Idea3, an innovative three-stage interaction paradigm:
- Idea Sharing: Each LLM proposes its initial solution and reasoning process.
- Idea Analysis: LLMs critically evaluate each other’s proposed methods. In cooperation mode, they identify complementary strengths. In competition mode, they rigorously assess merits and limitations.
- Idea Fusion: Insights from the analysis are synthesized into a refined and potentially novel solution. This stage aims to integrate the most effective elements from both approaches.
This structured interaction not only enhances the quality of the final response but also fosters a dynamic learning environment among LLMs, driving continuous improvement in their problem-solving capabilities.
Perception Calibration: Automated Reward Shaping
The second key component, Perception Calibration, addresses how LLMs learn from these interactions. Previous work has shown that tailored reward signals can guide LLMs towards better multi-agent learning. However, these often rely on manually predefined hyperparameters. ILR introduces a fully automated mechanism that integrates the reward distribution characteristics of one LLM’s responses into another LLM’s reward function. This allows each model to perceive the quality of its peers’ answers and generates continuous, fine-grained incentive signals without human intervention. The framework then uses Group Relative Policy Optimization (GRPO) to update each LLM based on these calibrated rewards.
Also Read:
- Adaptive Learning: How On-Demand Expert Help Boosts AI Reasoning
- Unlocking Continuous Learning in AI Agents with ReasoningBank
Experimental Validation and Key Findings
The effectiveness of ILR was evaluated on three LLMs (Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Qwen2.5-14B-Instruct) across various mathematical reasoning benchmarks (GSM8K, MATH-500, Minerva Math, Olympiad Bench, AIME24&25) and a code generation benchmark (MBPP). The results consistently showed that ILR outperforms traditional single-agent learning, yielding improvements of up to 5% over the strongest baselines. For instance, Llama-3.1-8B-Instruct trained with ILR improved by 3.12% over the strongest baseline GRPO.
Further analysis revealed two significant findings:
- Idea3 Enhances Robustness: The Idea3 communication paradigm enhances the robustness of stronger LLMs during multi-agent inference. By analyzing and reflecting on peer solutions, stronger LLMs are less likely to be misled by lower-quality responses from weaker LLMs.
- Dynamic Interaction is Superior: Dynamically determining interaction types (cooperation or competition) boosts the efficacy of multi-agent learning, outperforming purely cooperative or competitive strategies. This highlights the importance of adapting interaction strategies to question difficulty.
This research marks a significant step in understanding how multi-agent learning can enhance an LLM’s individual reasoning capabilities, moving beyond simply improving system-level performance. By drawing inspiration from human interaction, ILR provides a framework for LLMs to learn more effectively from each other, ultimately leading to more robust and independently capable models. You can read the full paper here.
