Prompt a Robot to Walk with Large Language Models
This addresses the problem of grounding LLMs in physical world tasks for robotics researchers, though it appears incremental as it builds on existing interest in LLMs for robotics.
The paper tackles the challenge of using large language models (LLMs) for robotics by introducing a method that generates low-level control commands for walking robots without task-specific fine-tuning, validated across various robots and environments.
Large language models (LLMs) pre-trained on vast internet-scale data have showcased remarkable capabilities across diverse domains. Recently, there has been escalating interest in deploying LLMs for robotics, aiming to harness the power of foundation models in real-world settings. However, this approach faces significant challenges, particularly in grounding these models in the physical world and in generating dynamic robot motions. To address these issues, we introduce a novel paradigm in which we use few-shot prompts collected from the physical environment, enabling the LLM to autoregressively generate low-level control commands for robots without task-specific fine-tuning. Experiments across various robots and environments validate that our method can effectively prompt a robot to walk. We thus illustrate how LLMs can proficiently function as low-level feedback controllers for dynamic motion control even in high-dimensional robotic systems. The project website and source code can be found at: https://prompt2walk.github.io/ .