TLDR: This paper introduces Embodied Questions Answering (EQsA), a new problem setting where embodied agents must handle multiple, asynchronous questions with varying urgencies in 3D environments. The authors propose ParaEQsA, a framework that uses a shared group memory to reduce redundant exploration and a priority-planning module to dynamically schedule questions based on urgency, scope, reward, and dependencies. They also contribute the PAEQs benchmark dataset and new evaluation metrics (DAR, NS, NUWL). ParaEQsA significantly outperforms sequential baselines in efficiency, responsiveness to urgent queries, and knowledge reuse, demonstrating the importance of parallel, urgency-aware scheduling for practical embodied agents.
Embodied agents, like the robots we envision helping us at home or in factories, are becoming increasingly sophisticated. A key capability for these agents is Embodied Question Answering (EQA), where a robot explores a 3D environment to find answers to questions. Traditionally, EQA systems have focused on answering one question at a time, which doesn’t quite reflect the dynamic and often chaotic nature of real-world human-robot interactions.
Imagine a rescue robot in a disaster zone. It might first be asked to locate a staircase, and then, while still exploring, receive an urgent follow-up question about trapped civilians. Or a factory assistant needing to monitor sensors, check inventory, and respond to safety alerts simultaneously. Current EQA systems struggle with these complex, multi-question scenarios where questions arrive at different times and have varying levels of importance.
To address this crucial gap, researchers Haisheng Wang and Weiming Zhi have introduced a new problem setting called Embodied Questions Answering (EQsA). This new framework challenges embodied agents to handle multiple questions that can arrive asynchronously and carry different urgencies. To tackle this, they propose a novel system named ParaEQsA: Parallel and Asynchronous Embodied Questions Scheduling and Answering.
ParaEQsA is designed to allow robots to process multiple questions concurrently, making them more efficient and responsive. A core innovation is its ‘group memory’ module, which is shared among all active questions. This means that information gathered while exploring for one question can be reused to answer others, significantly reducing the need for redundant exploration. Think of it as a shared knowledge base that all questions can tap into.
Another key component is the ‘priority-planning module’. This module dynamically schedules questions based on several factors: their inherent urgency (e.g., safety-related questions are more urgent), their ‘scope’ (whether they require a quick local observation or extensive global exploration), the ‘reward’ (how many other questions might benefit from answering this one), and any ‘dependencies’ (if one question needs another to be answered first). This intelligent scheduling ensures that the most critical and impactful questions are addressed promptly.
The ParaEQsA framework operates like a distributed system, with specialized modules working together. When a question comes in, a ‘Parser’ analyzes it for urgency and scope. A ‘Finishing Module’ first checks the shared ‘Group Memory’ to see if the question can be answered immediately without further exploration. If not, it goes to the ‘Question Pool’, where the ‘Planner’ selects the highest-priority question for targeted exploration. An ‘Answering Module’ then uses Vision-Language Models (VLMs) and the gathered context to provide the final answer.
To properly evaluate this new problem, the researchers also contributed a new benchmark dataset called Parallel Asynchronous Embodied Questions (PAEQs). This dataset is built using 40 photo-realistic indoor scenes and includes 5 questions per scene, totaling 200 questions. These questions are designed to be asynchronous, with initial questions and follow-up questions arriving with delays, and each question is assigned an urgency level (low, medium, or high) based on its semantic content (general, functional, or safety-related). The questions cover various categories like Existence, Counting, State, Identification, and Location.
New evaluation metrics were also introduced to measure performance beyond just accuracy. These include the Direct Answer Rate (DAR), which measures how often questions are answered directly from memory; Normalized Steps (NS), which quantifies exploration efficiency; and Normalized Urgency-Weighted Latency (NUWL), a crucial metric that assesses how quickly urgent questions are answered, reflecting the system’s responsiveness.
Experiments showed that ParaEQsA significantly outperforms traditional sequential EQA baselines. It achieved a much lower Normalized Steps (0.321 vs. 0.410 for Memory-EQA and 0.472 for Explore-EQA), indicating greater exploration efficiency. More impressively, its Normalized Urgency-Weighted Latency was dramatically reduced (0.204 vs. 0.474 and 0.551), demonstrating superior responsiveness to urgent queries. The system also achieved a 9% Direct Answer Rate, proving its ability to reuse knowledge effectively, something the baselines couldn’t do.
An ablation study further confirmed that each component of the priority planning mechanism – urgency, scope, reward, and dependency – plays a vital role in ParaEQsA’s overall performance. This research highlights that moving from a single-question, sequential approach to a parallel, urgency-aware scheduling paradigm with shared memory is essential for creating more capable and practical embodied agents for real-world applications. For more details, you can read the full research paper here.
Also Read:
- Evaluating AI Agents: Introducing MCP-AgentBench for Real-World Tool Use
- Optimizing Robotic AI: A Deep Dive into VLA Model Performance on Edge and Cloud Hardware
Looking ahead, the researchers suggest extending ParaEQsA to a multi-agent setup, where multiple robots could collaboratively explore and answer questions, potentially leading to even greater efficiency and responsiveness in complex environments.
