Towards Learning to Detect and Predict Contact Events on Vision-based Tactile Sensors
This work addresses the challenge of improving robotic grasping adaptability to external disturbances, though it is incremental as it applies deep learning to a known bottleneck in tactile sensing.
The researchers tackled the problem of classifying spatiotemporal tactile signals for robotic grasping by developing a deep learning framework that processes tactile image sequences from a vision-based sensor. They achieved a 52% increase in object lifting success rate with contact detection and significantly higher robustness under unexpected loads with slip prediction compared to open-loop grasps.
In essence, successful grasp boils down to correct responses to multiple contact events between fingertips and objects. In most scenarios, tactile sensing is adequate to distinguish contact events. Due to the nature of high dimensionality of tactile information, classifying spatiotemporal tactile signals using conventional model-based methods is difficult. In this work, we propose to predict and classify tactile signal using deep learning methods, seeking to enhance the adaptability of the robotic grasp system to external event changes that may lead to grasping failure. We develop a deep learning framework and collect 6650 tactile image sequences with a vision-based tactile sensor, and the neural network is integrated into a contact-event-based robotic grasping system. In grasping experiments, we achieved 52% increase in terms of object lifting success rate with contact detection, significantly higher robustness under unexpected loads with slip prediction compared with open-loop grasps, demonstrating that integration of the proposed framework into robotic grasping system substantially improves picking success rate and capability to withstand external disturbances.