Learning Tactile Models for Factor Graph-based Estimation
This work provides a more robust and scalable method for estimating object states from tactile feedback during robotic manipulation, which is important for robotics researchers and engineers working on dexterous manipulation tasks.
This paper addresses the problem of estimating object poses from touch during planar pushing under occlusions. The authors propose a two-stage approach: first, they learn local tactile observation models to predict the relative pose of the sensor from tactile images, and then integrate these models into a factor graph optimizer alongside physics and geometric factors. This method achieves reliable object tracking using only tactile feedback across 150 real-world planar pushing sequences with varying trajectories and three object shapes.
We're interested in the problem of estimating object states from touch during manipulation under occlusions. In this work, we address the problem of estimating object poses from touch during planar pushing. Vision-based tactile sensors provide rich, local image measurements at the point of contact. A single such measurement, however, contains limited information and multiple measurements are needed to infer latent object state. We solve this inference problem using a factor graph. In order to incorporate tactile measurements in the graph, we need local observation models that can map high-dimensional tactile images onto a low-dimensional state space. Prior work has used low-dimensional force measurements or engineered functions to interpret tactile measurements. These methods, however, can be brittle and difficult to scale across objects and sensors. Our key insight is to directly learn tactile observation models that predict the relative pose of the sensor given a pair of tactile images. These relative poses can then be incorporated as factors within a factor graph. We propose a two-stage approach: first we learn local tactile observation models supervised with ground truth data, and then integrate these models along with physics and geometric factors within a factor graph optimizer. We demonstrate reliable object tracking using only tactile feedback for 150 real-world planar pushing sequences with varying trajectories across three object shapes. Supplementary video: https://youtu.be/y1kBfSmi8w0