Realtime State Estimation with Tactile and Visual sensing. Application to Planar Manipulation
This addresses the challenge of robust manipulation for robots when visual feedback is lost, though it is incremental as it adapts existing SLAM methods to a specific scenario.
The paper tackles the problem of object state estimation in cluttered or occluded environments by integrating tactile and visual sensing using the iSAM framework, achieving real-time performance with improved accuracy in a pusher-slider system.
Accurate and robust object state estimation enables successful object manipulation. Visual sensing is widely used to estimate object poses. However, in a cluttered scene or in a tight workspace, the robot's end-effector often occludes the object from the visual sensor. The robot then loses visual feedback and must fall back on open-loop execution. In this paper, we integrate both tactile and visual input using a framework for solving the SLAM problem, incremental smoothing and mapping (iSAM), to provide a fast and flexible solution. Visual sensing provides global pose information but is noisy in general, whereas contact sensing is local, but its measurements are more accurate relative to the end-effector. By combining them, we aim to exploit their advantages and overcome their limitations. We explore the technique in the context of a pusher-slider system. We adapt iSAM's measurement cost and motion cost to the pushing scenario, and use an instrumented setup to evaluate the estimation quality with different object shapes, on different surface materials, and under different contact modes.