TLDR: DEXOP is a passive hand exoskeleton that introduces ‘perioperation,’ a new method for collecting high-quality, dexterous human manipulation data for robots. It mechanically links human and robot fingers, providing natural force feedback and mirroring human movements. This system significantly improves data collection efficiency and quality compared to traditional teleoperation, leading to more robust and generalizable robot learning policies for complex tasks.
Achieving human-level dexterity in robots remains one of the most significant challenges in robotics. While machine learning has made strides, the need for vast amounts of high-quality data often creates a bottleneck. Traditional methods like simulation, human activity videos, and teleoperation each have their drawbacks, from the “sim-to-real” gap to the lack of haptic feedback in teleoperation, which can make demonstrations unnatural and slow.
A new approach, termed “perioperation,” is emerging to address these issues. This paradigm focuses on sensorizing and recording human manipulation in a way that maximizes the transferability of skills to real robots. At the forefront of this innovation is DEXOP, a passive hand exoskeleton designed to facilitate this process.
DEXOP works by mechanically connecting human fingers to passive robot fingers. As a human user manipulates objects, DEXOP mirrors their hand pose to the passive robot hand and provides direct contact feedback through proprioception. This makes task demonstrations feel more natural and intuitive than traditional teleoperation, leading to increased speed and accuracy in data collection.
The design of DEXOP is guided by three key objectives. First, it aims to make data collection natural and scalable by offering high force transparency and kinematic coupling, eliminating the need for complex visual pose correction. Second, it ensures high transferability of collected data by separating the human hand from the passive robotic hand. This allows for co-design of the passive and actual robotic hands, enabling the integration of advanced sensors like whole-hand tactile sensing, crucial for capturing detailed force and contact information. Third, DEXOP enhances the diversity of accomplishable tasks through mechanical enhancements like fingernails for small object manipulation, abduction joints for in-hand reorientation, and a padded palm for stable whole-hand grasps.
Experiments have shown DEXOP’s superior performance compared to teleoperation. In user studies involving tasks like drilling, bulb installation, box packaging, and bottle opening, participants achieved significantly higher task throughput with DEXOP. For instance, in the drilling task, which proved nearly impossible with teleoperation, users completed it an average of six times per minute with DEXOP. This highlights DEXOP’s ability to provide the necessary proprioceptive feedback for complex, contact-rich tasks.
Beyond efficiency, DEXOP also contributes to better robot learning. Policies trained with data collected using DEXOP, even when mixed with a smaller amount of teleoperation data, consistently outperformed policies trained solely on teleoperation data. This is partly because the rich force feedback provided by DEXOP leads to cleaner, less biased demonstrations, reflecting more successful human strategies and improving policy generalization.
The researchers developed three variants of DEXOP: DEXOP-12 (four fingers, 12 degrees of freedom), DEXOP-9 (three fingers, nine degrees of freedom), and DEXOP-7 (three fingers, seven degrees of freedom), with DEXOP-7 being co-designed with the EyeSight Hand for direct skill transfer to a real robot. The system incorporates advanced tactile sensors, such as GelSim(ple) camera-based sensors, embedded in the fingertips, palm, and proximal phalanges to capture comprehensive contact information.
Also Read:
- Enhanced Control for Soft Robots: A Dual-Phase Reinforcement Learning Approach
- Robots Learn from One Example: A New Approach to Skill Generalization
While DEXOP represents a significant leap forward, the researchers acknowledge limitations and future work, including refining joint torque estimation, expanding the degrees of freedom in co-designed robotic hands, and integrating tactile feedback for human users. Nevertheless, DEXOP lays a strong foundation for scalable, real-world robotic data collection, paving the way for more dexterous and capable robots. You can find more details about this innovative system in the full research paper: DEXOP: A Device for Robotic Transfer of Dexterous Human Manipulation.
