IFG: Internet-Scale Guidance for Functional Grasping Generation
This addresses the challenge of precise robotic grasping in cluttered scenes, which is crucial for robotics applications, though it appears incremental as it builds on existing vision models and simulation techniques.
The paper tackles the problem of generating precise 3D grasps for robotic hands by combining internet-scale vision models with simulation-based geometric understanding, achieving high-performance semantic grasping without manually collected training data.
Large Vision Models trained on internet-scale data have demonstrated strong capabilities in segmenting and semantically understanding object parts, even in cluttered, crowded scenes. However, while these models can direct a robot toward the general region of an object, they lack the geometric understanding required to precisely control dexterous robotic hands for 3D grasping. To overcome this, our key insight is to leverage simulation with a force-closure grasping generation pipeline that understands local geometries of the hand and object in the scene. Because this pipeline is slow and requires ground-truth observations, the resulting data is distilled into a diffusion model that operates in real-time on camera point clouds. By combining the global semantic understanding of internet-scale models with the geometric precision of a simulation-based locally-aware force-closure, \our achieves high-performance semantic grasping without any manually collected training data. For visualizations of this please visit our website at https://ifgrasping.github.io/