TLDR: The IMAGINE framework integrates the reasoning and planning strengths of Multi-Agent Systems (MAS) into a single, smaller language model. It addresses MAS limitations like high costs and latency by using a three-stage training process (new query generation, MAS-based data generation, and agentic reasoning training). IMAGINE significantly boosts performance on complex tasks like travel planning, achieving an 82.7% Final Pass Rate on the TravelPlanner benchmark with a Qwen3-8B-Instruct base, surpassing larger models and even the original MAS.
Large Language Models (LLMs) have made incredible progress in understanding and generating human-like text. However, when it comes to tackling really complex tasks that require intricate reasoning and planning, they often hit a wall. Imagine trying to plan a detailed multi-day trip with many constraints – current LLMs, even advanced ones like GPT-4o, struggle significantly, achieving very low success rates on benchmarks like TravelPlanner.
To overcome these limitations, researchers have often turned to Multi-Agent Systems (MAS). These systems involve multiple LLM agents, each with a specific role, working together to solve a problem. While MAS can offer better collective reasoning, they come with their own set of problems: high computational costs due to many back-and-forth interactions, slow response times, and difficulty in training them efficiently from start to finish.
Introducing IMAGINE: A New Framework
A new framework called IMAGINE, which stands for “Integrating Multi-Agent System into One Model,” proposes a solution to these challenges. The core idea behind IMAGINE is to take the powerful reasoning and planning abilities typically found in a complex multi-agent system and distill them into a single, more compact, and efficient language model. This approach not only aims to match the performance of MAS but to significantly surpass it through a straightforward end-to-end training process.
The IMAGINE framework works in three main stages:
1. New Query Generation: Since existing datasets for complex planning tasks are often small, IMAGINE starts by creating a large number of diverse new queries. These queries are designed to cover various travel durations and difficulty levels, ensuring the model is exposed to a wide range of scenarios during training.
2. Multi-Agent System-based Inference Data Generation: The newly generated queries are then fed into a specially designed Multi-Agent System. This MAS, which includes roles like a Reasoner, Judges, and a Reflector, generates high-quality reasoning steps and final answers. This process effectively captures the collective intelligence and self-correction mechanisms of a team of agents.
3. Agentic Reasoning Training: This is where the magic happens. The data generated by the MAS is used to train a single, smaller language model. This training involves two key components: Agentic Supervised Fine-Tuning (SFT) to initially inject the MAS’s reasoning capabilities, followed by Agentic Reinforcement Learning (RL) to further enhance and refine the model’s planning abilities through end-to-end optimization.
Also Read:
- KVCOMM: Boosting Multi-Agent LLM Efficiency with Smart Memory Reuse
- Optimizing LLM Reasoning: A Hybrid Approach with Small and Large Models
Key Advantages and Performance
IMAGINE offers several significant benefits. By consolidating the capabilities into a single model, it eliminates the need for complex, multi-turn interactions between different agents. This drastically reduces user wait times, lowers inference costs, and requires less storage space. Furthermore, the end-to-end training makes it much easier to improve the model’s performance.
The experimental results are quite impressive. When using Qwen3-8B-Instruct as the base model and training it with the IMAGINE method, the model achieved an 82.7% Final Pass Rate on the challenging TravelPlanner benchmark. This is a remarkable improvement compared to other models, including DeepSeek-R1-671B (40%) and even the carefully designed Multi-Agent System (45.8%) that was used to generate the training data. This demonstrates that a single, smaller model can not only acquire but also significantly outperform a well-organized multi-agent team.
This research marks a significant step forward in developing more powerful and efficient AI models for complex reasoning and planning. By integrating the strengths of multi-agent systems into a compact, single model, IMAGINE paves the way for AI that can handle intricate tasks with greater accuracy and at a lower cost. You can read the full research paper here.
