TACTO: A Fast, Flexible, and Open-source Simulator for High-Resolution Vision-based Tactile Sensors
This simulator addresses the challenge of accurately simulating touch sensing, which is crucial for prototyping, debugging, and benchmarking new advances in robotics and learning for control, particularly for machine learning practitioners interested in multi-modal learning and control.
The paper introduces TACTO, an open-source simulator for high-resolution vision-based tactile sensors, capable of rendering realistic touch readings at hundreds of frames per second. It was demonstrated on a perceptual task, learning grasp stability from 1 million grasps, and a marble manipulation control task, also showing proof-of-concept for Sim2Real applications.
Simulators perform an important role in prototyping, debugging, and benchmarking new advances in robotics and learning for control. Although many physics engines exist, some aspects of the real world are harder than others to simulate. One of the aspects that have so far eluded accurate simulation is touch sensing. To address this gap, we present TACTO - a fast, flexible, and open-source simulator for vision-based tactile sensors. This simulator allows to render realistic high-resolution touch readings at hundreds of frames per second, and can be easily configured to simulate different vision-based tactile sensors, including DIGIT and OmniTact. In this paper, we detail the principles that drove the implementation of TACTO and how they are reflected in its architecture. We demonstrate TACTO on a perceptual task, by learning to predict grasp stability using touch from 1 million grasps, and on a marble manipulation control task. Moreover, we provide a proof-of-concept that TACTO can be successfully used for Sim2Real applications. We believe that TACTO is a step towards the widespread adoption of touch sensing in robotic applications, and to enable machine learning practitioners interested in multi-modal learning and control. TACTO is open-source at https://github.com/facebookresearch/tacto.