TLDR: TACOS is a new framework that uses Large Language Models (LLMs) to enable natural language control of multi-drone systems. It features a Coordinator LLM for high-level task planning and a Supervisor LLM for real-time execution, allowing a single pilot to manage complex drone swarms through intuitive commands, reducing workload and improving mission efficiency.
Managing multiple drones simultaneously can be a complex challenge, especially when a single human pilot is in charge. The demands on the pilot can range from directly controlling individual drones to coordinating entire groups or even managing fully autonomous drone swarms for high-level tasks. To address this, researchers Alessandro Nazzari, Roberto Rubinacci, and Marco Lovera have introduced TACOS (Task-Agnostic COordinator of a multi-drone System), a new framework designed to simplify the control of multi-drone systems using natural language.
TACOS aims to reduce pilot workload by allowing high-level task delegation through intuitive, language-based interfaces. It integrates three core capabilities into a single architecture: a natural language interface for easy user interaction, an intelligent coordinator that translates user commands into structured task plans, and an autonomous agent that executes these plans by interacting with the real world. Essentially, TACOS enables a Large Language Model (LLM) to communicate with a library of executable commands, bridging the gap between human language and real-time multi-robot coordination.
The framework is built around two main language models in a hierarchical structure: the Coordinator LLM and the Supervisor LLM. The Coordinator receives high-level natural language commands from the user and creates a task plan. The Supervisor then takes this plan and sequences and executes it based on the real-time status of the drones and their environment. Each LLM is given specific instructions to define its objectives and how it should behave.
The Coordinator’s role is to convert user instructions, like “Split the swarm into two groups, send one group north, and have the other surround the target,” into a structured list of API calls. It also provides a natural language explanation of its reasoning, which helps in understanding how the plan was generated. To improve its planning abilities, the Coordinator uses techniques like In-Context Learning (ICL) and Chain of Thought (COT) prompting, where it learns from examples and explicitly reasons through its decisions.
The Supervisor LLM, on the other hand, is responsible for turning the Coordinator’s high-level plan into a sequence of executable actions. It operates in a continuous feedback loop, receiving updated information from the drones and the environment every few seconds. This allows it to adjust actions, ensure dependencies are met, and assign tasks to individual drones. While the Coordinator has access to the full history of user interactions to maintain context, the Supervisor has a more limited, task-specific memory that is cleared once a task is completed.
Experiments were conducted in a simulated urban environment and with real-world multi-drone systems. The evaluation showed that TACOS is effective in managing complex tasks. For instance, in a “find the dog” mission, TACOS intelligently prioritized searching parks over business districts based on semantic reasoning, demonstrating its ability to make contextually aware decisions. The system also efficiently assigned search tasks to the closest available drones and adapted to new instructions, such as monitoring the dog and notifying its owner.
An ablation study, which involved testing TACOS with certain modules removed, highlighted the benefits of its modular design. The full TACOS framework, with its dedicated reasoning (Coordinator) and execution (Supervisor) agents, proved more efficient and successful for tasks requiring complex planning compared to configurations where these roles were combined or reasoning was omitted. The researchers believe that incorporating onboard perception and expanding the set of available APIs will further enhance TACOS’s capabilities.
Also Read:
- AI-Powered Robot Teams Master Exploration and Search in Unknown Environments
- Autonomous Drones Get Smarter: A Hierarchical Framework for Industrial Inspection
This work represents a significant step towards more flexible and resilient swarm mission execution in unpredictable settings, offering a powerful new way for humans to interact with and control multi-drone systems. For more technical details, you can read the full research paper here.
