ThinkGrasp: A Vision-Language System for Strategic Part Grasping in Clutter
This addresses the problem of robotic grasping in heavy clutter for robotics applications, representing a novel method for a known bottleneck.
The paper tackles robotic grasping in cluttered environments by developing ThinkGrasp, a vision-language system that uses GPT-4o's reasoning to guide object removal and achieve high success rates in both simulated and real experiments, outperforming state-of-the-art methods.
Robotic grasping in cluttered environments remains a significant challenge due to occlusions and complex object arrangements. We have developed ThinkGrasp, a plug-and-play vision-language grasping system that makes use of GPT-4o's advanced contextual reasoning for heavy clutter environment grasping strategies. ThinkGrasp can effectively identify and generate grasp poses for target objects, even when they are heavily obstructed or nearly invisible, by using goal-oriented language to guide the removal of obstructing objects. This approach progressively uncovers the target object and ultimately grasps it with a few steps and a high success rate. In both simulated and real experiments, ThinkGrasp achieved a high success rate and significantly outperformed state-of-the-art methods in heavily cluttered environments or with diverse unseen objects, demonstrating strong generalization capabilities.