Monocular Reconstruction of Neural Tactile Fields
This addresses the challenge for robots to plan paths in deformable environments by avoiding high-resistance objects and routing through low-resistance regions, representing a novel method for a known bottleneck.
The paper tackled the problem of robots needing interaction-aware 3D representations for deformable environments by introducing neural tactile fields, which map spatial locations to expected tactile responses from a single monocular image, resulting in improvements of 85.8% in volumetric 3D reconstruction and 26.7% in surface reconstruction compared to state-of-the-art methods.
Robots operating in the real world must plan through environments that deform, yield, and reconfigure under contact, requiring interaction-aware 3D representations that extend beyond static geometric occupancy. To address this, we introduce neural tactile fields, a novel 3D representation that maps spatial locations to the expected tactile response upon contact. Our model predicts these neural tactile fields from a single monocular RGB image -- the first method to do so. When integrated with off-the-shelf path planners, neural tactile fields enable robots to generate paths that avoid high-resistance objects while deliberately routing through low-resistance regions (e.g. foliage), rather than treating all occupied space as equally impassable. Empirically, our learning framework improves volumetric 3D reconstruction by $85.8\%$ and surface reconstruction by $26.7\%$ compared to state-of-the-art monocular 3D reconstruction methods (LRM and Direct3D).