PerAct2: Benchmarking and Learning for Robotic Bimanual Manipulation Tasks
This provides a benchmark for researchers studying bimanual manipulation, though it is incremental as it builds on existing frameworks.
The paper tackles the lack of simulated benchmarks for bimanual robotic manipulation by extending RLBench to include 13 new tasks with 23 variations, and introduces PerAct2, a language-conditioned agent that enables learning and execution of these tasks.
Bimanual manipulation is challenging due to precise spatial and temporal coordination required between two arms. While there exist several real-world bimanual systems, there is a lack of simulated benchmarks with a large task diversity for systematically studying bimanual capabilities across a wide range of tabletop tasks. This paper addresses the gap by extending RLBench to bimanual manipulation. We open-source our code and benchmark comprising 13 new tasks with 23 unique task variations, each requiring a high degree of coordination and adaptability. To kickstart the benchmark, we extended several state-of-the art methods to bimanual manipulation and also present a language-conditioned behavioral cloning agent -- PerAct2, which enables the learning and execution of bimanual 6-DoF manipulation tasks. Our novel network architecture efficiently integrates language processing with action prediction, allowing robots to understand and perform complex bimanual tasks in response to user-specified goals. Project website with code is available at: http://bimanual.github.io