Large Language Models for Orchestrating Bimanual Robots
This work addresses the problem of effective temporal and spatial coordination for bimanual robots, which is crucial for advancing robotic manipulation capabilities, though it appears incremental as it builds on existing LLM-based approaches.
The paper tackles the challenge of generating control policies for bimanual robots to solve long-horizon tasks by introducing LABOR, an agent that uses a Large Language Model to analyze tasks and devise coordination policies, and it demonstrates improved success rates over baselines in simulated experiments with the NICOL humanoid robot.
Although there has been rapid progress in endowing robots with the ability to solve complex manipulation tasks, generating control policies for bimanual robots to solve tasks involving two hands is still challenging because of the difficulties in effective temporal and spatial coordination. With emergent abilities in terms of step-by-step reasoning and in-context learning, Large Language Models (LLMs) have demonstrated promising potential in a variety of robotic tasks. However, the nature of language communication via a single sequence of discrete symbols makes LLM-based coordination in continuous space a particular challenge for bimanual tasks. To tackle this challenge, we present LAnguage-model-based Bimanual ORchestration (LABOR), an agent utilizing an LLM to analyze task configurations and devise coordination control policies for addressing long-horizon bimanual tasks. We evaluate our method through simulated experiments involving two classes of long-horizon tasks using the NICOL humanoid robot. Our results demonstrate that our method outperforms the baseline in terms of success rate. Additionally, we thoroughly analyze failure cases, offering insights into LLM-based approaches in bimanual robotic control and revealing future research trends. The project website can be found at http://labor-agent.github.io.