Natural Language as Policies: Reasoning for Coordinate-Level Embodied Control with LLMs
This work addresses robotics task planning for researchers and practitioners by offering a more direct method that reduces reliance on pre-defined APIs, though it appears incremental as it builds on existing LLM applications in robotics.
The paper tackles robotics task planning by using LLMs to directly output coordinate-level control commands from natural language descriptions, eliminating the need for intermediate code. It shows that this approach significantly improves success rates on a simulation benchmark and enables skill transfer to unseen tasks.
We demonstrate experimental results with LLMs that address robotics task planning problems. Recently, LLMs have been applied in robotics task planning, particularly using a code generation approach that converts complex high-level instructions into mid-level policy codes. In contrast, our approach acquires text descriptions of the task and scene objects, then formulates task planning through natural language reasoning, and outputs coordinate level control commands, thus reducing the necessity for intermediate representation code as policies with pre-defined APIs. Our approach is evaluated on a multi-modal prompt simulation benchmark, demonstrating that our prompt engineering experiments with natural language reasoning significantly enhance success rates compared to its absence. Furthermore, our approach illustrates the potential for natural language descriptions to transfer robotics skills from known tasks to previously unseen tasks. The project website: https://natural-language-as-policies.github.io/