HumanoidGen: Data Generation for Bimanual Dexterous Manipulation via LLM Reasoning
This addresses a domain-specific problem for robotics researchers by providing a novel data generation method for humanoid robots, though it is incremental as it builds on existing LLM and planning techniques.
The paper tackles the lack of simulation tasks and demonstrations for bimanual dexterous manipulation in humanoid robots by introducing HumanoidGen, an automated framework that uses LLM reasoning and atomic operations to generate data, resulting in improved performance for 2D and 3D diffusion policies that scales with the generated dataset.
For robotic manipulation, existing robotics datasets and simulation benchmarks predominantly cater to robot-arm platforms. However, for humanoid robots equipped with dual arms and dexterous hands, simulation tasks and high-quality demonstrations are notably lacking. Bimanual dexterous manipulation is inherently more complex, as it requires coordinated arm movements and hand operations, making autonomous data collection challenging. This paper presents HumanoidGen, an automated task creation and demonstration collection framework that leverages atomic dexterous operations and LLM reasoning to generate relational constraints. Specifically, we provide spatial annotations for both assets and dexterous hands based on the atomic operations, and perform an LLM planner to generate a chain of actionable spatial constraints for arm movements based on object affordances and scenes. To further improve planning ability, we employ a variant of Monte Carlo tree search to enhance LLM reasoning for long-horizon tasks and insufficient annotation. In experiments, we create a novel benchmark with augmented scenarios to evaluate the quality of the collected data. The results show that the performance of the 2D and 3D diffusion policies can scale with the generated dataset. Project page is https://openhumanoidgen.github.io.