TLDR: Auras is a new framework that significantly speeds up embodied AI agents by separating their perception and action generation processes and running them in parallel. It uses a shared “public context” to ensure decisions are always based on the latest information, achieving 2.54x higher throughput with 102.7% of original accuracy, making real-time AI interactions more efficient.
Embodied Artificial Intelligence (AI) systems, which interact with the physical world in real-time, face a significant challenge: achieving a high “thinking” frequency. Unlike chatbots that operate in virtual environments, embodied AI agents, used in robotics and autonomous driving, need to process continuous sensory input and generate actions at very high speeds. Traditional methods, which process perception (understanding the environment) and generation (deciding actions) sequentially, often lead to slow response times, hindering their effectiveness in real-world applications.
Researchers have introduced a novel framework called Auras, an algorithm-system co-designed inference framework, to tackle this problem. Auras aims to significantly boost the inference frequency of embodied AI agents without sacrificing accuracy. The core idea behind Auras is to break down the rigid sequential computation pattern by disaggregating the perception and generation modules.
Auras achieves this by allowing the perception and generation modules to operate with controlled pipeline parallelism. This means that instead of waiting for one process to fully complete before the next begins, multiple stages of perception and generation can run concurrently. A key innovation is the establishment of a “public context” – a shared buffer where the latest environmental information from the perception module is stored. The generation module can then asynchronously fetch and compute based on this fresh data, even if it’s working on a different stage of a parallel request. This ingenious design addresses the “data staleness” problem, where parallel processing might otherwise lead to decisions based on outdated information, thus maintaining high accuracy.
The framework also includes an asynchronous pipeline executor that intelligently coordinates the execution of these parallel stages. This executor uses a hierarchical tuning process to find the optimal configuration for pipelining both perception and generation, balancing the need for high throughput with the necessity of maintaining agent accuracy. For instance, it can adjust how many stages are in the perception and generation pipelines and how computational tasks are distributed among them.
Experimental results demonstrate the remarkable efficacy of Auras. On average, it improves the throughput of embodied AI systems by 2.54 times, meaning agents can “think” and act much faster. Crucially, this performance boost comes with virtually no loss in accuracy, achieving 102.7% of the original accuracy compared to traditional sequential methods. This highlights Auras’ capability to overcome the limitations of sequential computation, providing a significant leap forward for real-world embodied AI applications.
Also Read:
- Orchestrator: Enhancing Multi-Agent AI Coordination in Complex, Long-Duration Tasks
- Enhancing AI Agents for Continuous Task Execution in Dynamic Environments
For more in-depth technical details, you can refer to the full research paper: Boosting Embodied AI Agents through Perception-Generation Disaggregation and Asynchronous Pipeline Execution.
