Beyond Visual Grasping: Benchmarking Complex Grasping from Detection to Execution
For robotic grasping researchers, this benchmark exposes the gap between existing grasp detection methods and the demands of real-world tasks requiring reasoning and semantic constraints.
The authors introduce GCA-Bench, a benchmark for complex robotic grasping that requires multi-step reasoning and semantic understanding, and show that current methods achieve success rates below 70% on these scenarios, highlighting critical limitations.
Robust robotic grasping remains a fundamental challenge for complex real-world applications. Recent advances in large-scale models demonstrate promising capabilities for reasoning in robotic tasks. However, existing benchmarks for grasping primarily focus on isolated, visual-based grasp pose detection, failing to capture the complexity of grasping tasks that require multi-step reasoning and semantic understanding during execution. To address this gap, we propose GCA-Bench, a benchmark featuring challenging \textit{grasping with complex action} scenarios that involve both scene-level reasoning and semantic constraints. GCA-Bench enables the evaluation of recent large foundation models under the same settings. To demonstrate the effectiveness of our new benchmark, we implement a diverse set of baselines, ranging from traditional grasp detection pipelines to end-to-end learning methods. Empirical studies achieve success rates below 70\% on complex grasping scenarios, underscoring critical limitations. In addition, we propose new evaluation metrics, analyze critical failure models, and provide insights to guide the development of more robust and generalizable grasping strategies.