TLDR: This research paper details how different prompting methods and Large Language Models (LLMs) can significantly improve cooperation and decision-making in multi-agent AI systems, specifically within the CoELA framework. By optimizing prompts and integrating speech capabilities, the study achieved a 22% efficiency improvement with Gemma 3, demonstrating the critical role of prompt engineering in enhancing collaborative agent performance and user interaction.
In the evolving landscape of artificial intelligence, the focus is increasingly shifting from individual intelligent agents to systems where AI entities can work together effectively. This vision is at the heart of Collaborative Embodied AI (CEAI), a field dedicated to creating virtual or physical agents that operate in shared environments, coordinating their actions to achieve common goals.
A recent paper, “Improving Cooperation in Collaborative Embodied AI,” explores how Large Language Models (LLMs) can significantly enhance this collaborative behavior. The research builds upon an existing framework called CoELA (Collaborative Embodied Language Agents), which is designed for multi-agent communication, reasoning, and task coordination in virtual spaces.
Enhancing Collaboration with LLMs and Prompt Engineering
The core of this research lies in systematically experimenting with different LLMs and various prompt engineering strategies. The goal was to identify optimal combinations that maximize collaboration performance among AI agents. The study found that prompt optimization is highly effective, with the best combination improving the efficiency of a system running with Gemma 3 by 22% compared to the original CoELA setup.
The CoELA framework itself is modular, with each agent operating independently through five interconnected modules: Perception, Memory, Planning, Communication, and Execution. This paper specifically focused on enhancing the Planning and Communication modules, which are powered by LLMs.
Flexible LLM Integration and Prompt Strategies
To allow for flexible experimentation with different LLMs, the researchers integrated Ollama, a lightweight local runtime environment. This made it easy to switch between models like Llama 3.1, DeepSeek r1, Mistral, and Gemma 3, without altering the core agent pipeline. These models were chosen to balance performance with hardware efficiency, ensuring the system could run efficiently on a single GPU.
The team developed and tested various prompting methods across three modules: Planning, Communication, and Action. For the Planning module, they introduced an “Explicit Instruction – Improved Baseline” prompt to encourage more goal-focused decisions and a “Structured/Forced Reasoning” prompt to guide LLMs through step-by-step analysis. In the Communication module, several prompts were tested, including those that removed verbose output, used one-shot examples, or multi-shot examples to refine message tone and structure. The Action prompt was also refined with explicit instructions and examples to ensure the LLM’s output aligned with system expectations.
Speech Integration for Enhanced Interaction
Beyond prompt optimization, the research extended to integrating speech capabilities, enabling seamless collaborative voice-based interactions. A Text-to-Speech (TTS) and Chat GUI module was developed, providing a synchronized visual and auditory interface for real-time dialogue between agents (named Alice and Bob). This feature not only aids in understanding system behavior but also supports iterative system development and demonstrations, making the interactions more engaging.
Key Findings and Efficiency Gains
The evaluation used metrics such as step count (number of script executions to complete a task) and turn count (number of dialogue turns between agents). The results showed that the combination of the “Improved Base” planning prompt and “CPrompt4” (which merged instruction removal with a one-shot example) consistently reduced both step count and turn count for models like Llama 3.1 and Gemma 3. For instance, Gemma 3 saw its step count decrease from 84.0 to 65.5, a significant improvement in execution efficiency.
A notable observation was that Gemma 3, despite being a smaller model (4B parameters), performed comparably to the larger Llama 3.1 (8B parameters), suggesting advancements in newer, more efficient models. The research also highlighted that while prompt engineering significantly improved communication clarity and task execution, some models, like DeepSeek r1, still tended to include internal reasoning in their responses despite restrictive prompts.
This work underscores the critical role of prompt engineering in boosting collaborative agent performance and demonstrates the value of integrating interactive, multimodal agent interfaces. The full research paper can be found here: Improving Cooperation in Collaborative Embodied AI.
Also Read:
- Collaborative Agents for Automated Data Visualization
- Google Unveils TUMIX: A Novel Multi-Agent Framework for Enhanced LLM Performance and Cost Efficiency
Future Directions
Looking ahead, the researchers plan to expand their evaluation to include human-agent interaction and ad-hoc team formations with more than two agents. This will help in understanding how agents respond to natural human communication and dynamic team structures, ultimately improving collaborative agent behavior in real-world, mixed-team settings.
