RealDex: Towards Human-like Grasping for Robotic Dexterous Hand
This work addresses the challenge of enabling humanoid robots to perform natural and precise grasping in real-world scenarios, representing an incremental advancement in robotic manipulation.
The paper tackles the problem of generating human-like grasping motions for robotic dexterous hands by introducing RealDex, a dataset with authentic human behavioral patterns and multi-view multimodal data, and a framework using Multimodal Large Language Models, showing superior performance in experiments.
In this paper, we introduce RealDex, a pioneering dataset capturing authentic dexterous hand grasping motions infused with human behavioral patterns, enriched by multi-view and multimodal visual data. Utilizing a teleoperation system, we seamlessly synchronize human-robot hand poses in real time. This collection of human-like motions is crucial for training dexterous hands to mimic human movements more naturally and precisely. RealDex holds immense promise in advancing humanoid robot for automated perception, cognition, and manipulation in real-world scenarios. Moreover, we introduce a cutting-edge dexterous grasping motion generation framework, which aligns with human experience and enhances real-world applicability through effectively utilizing Multimodal Large Language Models. Extensive experiments have demonstrated the superior performance of our method on RealDex and other open datasets. The complete dataset and code will be made available upon the publication of this work.