Creating and Repairing Robot Programs in Open-World Domains
This addresses the issue of robot program failures in open-world domains for robotics and AI researchers, but it is incremental as it builds on existing LLM-based methods.
The paper tackles the problem of faulty LLM-generated robot programs by proposing RoboRepair, a system that traces execution to errors and uses LLM-produced recovery programs to minimize repeated actions, achieving efficiency comparable to an oracle with foreknowledge in a benchmark of eleven tasks.
Using Large Language Models (LLMs) to produce robot programs from natural language has allowed for robot systems that can complete a higher diversity of tasks. However, LLM-generated programs may be faulty, either due to ambiguity in instructions, misinterpretation of the desired task, or missing information about the world state. As these programs run, the state of the world changes and they gather new information. When a failure occurs, it is important that they recover from the current world state and avoid repeating steps that they they previously completed successfully. We propose RoboRepair, a system which traces the execution of a program up until error, and then runs an LLM-produced recovery program that minimizes repeated actions. To evaluate the efficacy of our system, we create a benchmark consisting of eleven tasks with various error conditions that require the generation of a recovery program. We compare the efficiency of the recovery program to a plan built with an oracle that has foreknowledge of future errors.