TLDR: The DeepXPalm system introduces a novel approach to improve telemanipulation of deformable objects by providing clear tactile feedback to the user’s palm. It uses a Robotiq gripper with tactile sensors and a palm-worn haptic display called LinkGlide. A Convolutional Neural Network (CNN) is employed to accurately detect the tilt and position of grasped objects, generating a “mask” to enhance the tactile stimuli. Experiments showed that using the CNN-based masking significantly increased user recognition of object tilt and position from 9.67% to 82.5%, making remote handling of deformable objects like plastic pipettes much more precise.
Telemanipulation, the ability to control robots remotely, is becoming increasingly important in various fields, from manufacturing to healthcare. However, handling delicate or deformable objects remotely presents a significant challenge. Users often struggle with accurately perceiving the object’s shape, tilt, and position, leading to errors and a lack of dexterity. This is where the innovative DeepXPalm system steps in, offering a solution that combines advanced haptic feedback with artificial intelligence.
The DeepXPalm system, detailed in the research paper DeepXPalm: Tilt and Position Rendering using Palm-worn Haptic Display and CNN-based Tactile Pattern Recognition, addresses this problem by providing users with clear and precise tactile feedback directly on their palm. The system is designed for telemanipulating deformable objects, such as plastic pipettes, which are commonly used in laboratories and require high precision.
How DeepXPalm Works
At its core, DeepXPalm integrates a multi-contact haptic device called LinkGlide, worn on the user’s palm, with a 2-finger Robotiq gripper equipped with tactile sensor arrays. When the robotic gripper grasps an object, these sensors detect the pressure distribution, providing crucial information about the object’s tilt and position.
The most significant innovation lies in the use of a Convolutional Neural Network (CNN). This AI algorithm processes the raw data from the tactile sensors. Instead of directly translating the sensor data to the user’s palm, which can be ambiguous, the CNN analyzes the data to accurately classify the object’s tilt angle and position. Based on this classification, the CNN generates a ‘mask’ – a predefined tactile pattern – which is then combined with the downsized sensor data. This masked data is what the LinkGlide device renders to the user’s palm.
The LinkGlide Haptic Display
The LinkGlide device is a wearable haptic display that provides touch sensations at three independent contact points on the user’s palm. It utilizes an array of inverted five-bar linkages, each controlled by two servo motors, to generate precise tactile stimuli. This multi-contact approach allows for a richer and more nuanced representation of the object’s interaction with the gripper.
Significant Improvement in Perception
To evaluate the effectiveness of DeepXPalm, two experiments were conducted. In the first, users received tactile feedback directly from the downsized sensor data. The results were poor, with an average recognition rate of only 9.67% for both tilt and position, and 28% for tilt alone. This highlights the difficulty in interpreting raw, complex tactile information.
However, when the CNN-based masking method was applied, the user’s perception dramatically improved. The overall recognition rate for object tilt and position soared to 82.5%. This demonstrates that the CNN’s ability to interpret and simplify complex tactile data into clear, recognizable patterns is highly effective in enhancing human perception during telemanipulation.
Also Read:
- TiltXter: Enhancing Remote Manipulation of Deformable Objects with CNN-Powered Tactile Feedback
- Advancing Robotic Grasping: A Unified Approach for Diverse Dexterous Hands
Applications and Future Potential
The DeepXPalm system holds immense potential for various applications, particularly in environments requiring precise handling of delicate objects. For instance, it could significantly improve the safety and efficiency of medical staff performing COVID tests or other laboratory procedures remotely using robotic systems. By providing intuitive and accurate tactile feedback, DeepXPalm can make telemanipulation more dexterous and reduce errors, paving the way for more advanced human-robot collaboration in remote settings.
