TLDR: A new robotics framework allows robots to learn and execute tasks using tools of varying lengths. It extends inverse kinematics to account for tool length, trains a policy in simulation, and transfers this learning to real-world robots, demonstrating robust performance with different tools and an impressive error rate of less than 1cm for the extended inverse kinematics solver and a mean error of 8cm in simulation for the trained policy.
Robots are becoming increasingly sophisticated, but their ability to use tools effectively has traditionally been limited. Conventional robots often rely on pre-programmed tasks and have a restricted understanding of their own kinematics, which is the study of motion without considering the forces that cause it. This limitation hinders their capacity to leverage tools efficiently, a skill that is fundamental for humans and even some animals.
<
A new framework addresses this challenge by expanding a robot’s inverse kinematics solver, enabling it to learn a sequence of actions using tools of different lengths. Inverse kinematics is a complex problem in robotics that involves calculating the joint angles required for a robot’s end-effector (like a gripper or tool tip) to reach a desired position and orientation in space.
A Novel Approach to Tool Manipulation
The research, conducted by Prathamesh Kothavale and Sravani Boddepalli, introduces a pioneering framework that focuses on the crucial aspect of tool usage: precise manipulation. Their approach is built on three core contributions:
- Extending an inverse kinematics model with an additional fixed joint to account for a picked-up tool.
- Developing a novel Baxter robot simulation model, calibrated against a physical system, for reinforcement learning.
- Creating an architecture that combines the extended inverse kinematics model with a learned policy, allowing the system to determine action trajectories robust to variable tool lengths.
- DynaRend: A New Framework for Robots to Learn 3D Dynamics
- Unlocking Robot Adaptability: How Parameterized Skills Enhance Learning from Demonstrations
The Baxter robot, a humanoid dual-armed robot from Rethink Robotics, was chosen for this research due to its human-like movement capacity and dexterity. It is controlled using the Robotic Operating System (ROS), which provides a flexible framework for robot software.
Detecting Tool Length and Adapting Movement
A key innovation of this framework is its ability to detect the length of a gripped tool using basic computer vision techniques. By programming the Baxter robot to move to three different orientations, the system captures images and uses HSV color masking to identify the gripper and tool tip. The pixel distance between these points is then scaled to determine the real-world tool length.
Once the tool’s length is known, the inverse kinematics model is extended to move the tool’s tip to any desired location. This is achieved by calculating an offset position for the gripper, effectively directing the tool tip. This ingenious method allows any valid path learned in simulation to be directly transferred to the physical robot, even if the real-world tool has a different length than the one used in training. This means the robot can use tools of arbitrary lengths without needing to be retrained.
Learning in Simulation
Training robots in the real world is time-consuming and resource-intensive. Therefore, the researchers utilized a simulation environment for learning. They modeled the Baxter robot in the MuJoCo physics engine and wrapped it into an OpenAI Gym robotics environment for reinforcement learning. This setup allowed for extensive training, which is critical for deep reinforcement learning models that require millions of data points.
The robot’s task in simulation was to use a variable-length tool to push a wooden box to a target location. The agent observed various parameters, including the goal box position, current box position, tool position, gripper position and orientation, and joint angles and velocities. Several reinforcement learning algorithms were tested, including A2C, TRPO, PPO, and DDPG. Among these, the Proximal Policy Optimization (PPO) algorithm proved to be the most successful, learning a stable policy to push the box towards the goal.
Also Read:
Bridging the Simulation-to-Reality Gap
After training in simulation, the learned action trajectories (sequences of 3D coordinates and quaternions for gripper position and orientation) were transferred to the physical Baxter robot. While a gap between simulation and real-world performance is expected, the PPO algorithm demonstrated successful transferability. The extended inverse kinematics model, combined with the learned policy, showed remarkable robustness to variable tool lengths.
Experiments revealed that the extended inverse kinematics solver achieved an impressive error rate of less than 1cm. In real-world tests, the model moved the box an average of 19.9cm for a longer 17.5cm tool and 19.7cm for a shorter 12.5cm tool. This performance, while slightly less than the 27.26cm achieved in simulation, highlights the model’s ability to perform virtually identically despite different tool lengths. This indicates that the learned policy is robust and can determine equivalent action trajectories for various tool sizes.
The researchers acknowledge limitations, such as the simulation not fully capturing tool pliancy or slippage, and the physical robot’s inverse kinematics solver being more conservative about reachable positions than the simulation. Future work aims to refine the simulator to better reflect reality and explore learning with different tool shapes and types.
This research marks a significant step towards enabling robots to master the intricate art of tool manipulation across diverse tasks, paving the way for more general-purpose robots that can adapt and learn with limited human assistance. You can read the full research paper here.
