TLDR: The AIRoA MoMa Dataset is a new large-scale, real-world dataset for mobile manipulation robots, featuring over 25,000 episodes of household tasks. It uniquely combines synchronized multimodal data, including force-torque sensing, with hierarchical annotations and explicit failure cases. Designed to overcome limitations of existing datasets, it enables robots to learn complex, contact-rich, and long-horizon tasks, pushing the development of general-purpose robotic agents.
The dream of general-purpose robots seamlessly operating in our homes, assisting with daily chores, is a step closer to reality thanks to a groundbreaking new resource: the AIRoA MoMa Dataset. Developed by a collaboration of leading institutions including The University of Tokyo, AI Robot Association (AIRoA), and Toyota Motor Corporation, this large-scale dataset is specifically designed to tackle the complex challenges of mobile manipulation in unstructured human environments.
For years, the development of intelligent robots has been hampered by limitations in available training data. Existing datasets often focus on simpler, fixed-base tasks like picking objects off a tabletop. They frequently lack crucial information about physical interaction, known as ‘contact-rich’ tasks, and rarely capture the long, multi-step sequences required for real-world activities like making coffee or doing laundry. These gaps have prevented robots from moving beyond basic pick-and-place scenarios to truly robust, real-world assistance.
The AIRoA MoMa Dataset directly addresses these shortcomings. It’s a massive collection of real-world data, comprising over 25,000 episodes and approximately 94 hours of robot operation. What makes it unique is its comprehensive approach to data collection and annotation.
What Makes AIRoA MoMa Stand Out?
Firstly, it focuses on mobile manipulation, meaning the robot isn’t stationary but navigates and interacts within a household setting. This is a significant leap from tabletop-only tasks, requiring the robot to integrate movement and dexterity.
Secondly, the dataset captures contact-rich interactions. For tasks like pressing a light switch or opening a drawer, visual information alone isn’t enough. AIRoA MoMa includes synchronized six-axis wrist force-torque signals, providing the robot with a sense of touch. This multimodal data – combining RGB images from two viewpoints (head and wrist), joint states, and force-torque feedback – is crucial for learning physically grounded interactions.
Thirdly, it emphasizes long-horizon tasks. Real-world activities are rarely single actions; they involve a sequence of steps. The dataset introduces a novel two-layer annotation scheme: high-level ‘Short Horizon Tasks’ (like ‘Bake a toast’) are broken down into a series of ‘Primitive Actions’ (like ‘Open Oven,’ ‘Pick Bread’). This hierarchical structure is vital for training robots to plan and execute complex, multi-step operations and also allows for detailed error analysis.
The dataset also includes explicit failure cases, which are invaluable for teaching robots how to detect and recover from errors, leading to more robust and resilient policies. All data is standardized in the widely adopted LeRobot v2.1 format, ensuring compatibility with existing Vision-Language-Action (VLA) models and fostering reproducibility across the research community.
Also Read:
- Adaptive Learning for Multitask Robot Policies
- OMNIRETARGET: Enhancing Humanoid Robot Learning Through Interaction-Aware Motion Generation
How Was the Data Collected?
The data was collected using the Toyota Human Support Robot (HSR), a versatile personal assistant robot. To ensure high-quality, complex behavioral data, a specialized one-to-one joint-mapping teleoperation system, THSR, was developed. This system allowed 18 trained human operators to intuitively control the HSR, performing various household tasks in a laboratory environment designed to replicate real homes, including kitchens, living rooms, and bathrooms. Object placement, lighting, and robot starting positions were randomized to enhance data diversity.
The AIRoA MoMa Dataset is a significant contribution to the field of robotics. By providing a rich, diverse, and meticulously annotated resource, it serves as a critical benchmark for advancing the next generation of VLA models. It promises to accelerate the development of general-purpose robotic agents capable of performing complex, contact-rich, and long-horizon tasks in our everyday environments. You can learn more about this work by reading the full research paper here: AIRoA MoMa Dataset: A Large-Scale Hierarchical Dataset for Mobile Manipulation.
