Ground Manipulator Primitive Tasks to Executable Actions using Large Language Models
This addresses the gap in robot layered architectures for grounding tasks to executable actions, specifically for manipulator systems.
The paper tackles the challenge of translating high-level manipulator primitive tasks into low-level robot actions by proposing a novel approach using large language models (LLMs) with program-function-like prompts based on task frame formalism, enabling LLMs to generate position/force set-points for hybrid control and providing evaluations over several state-of-the-art LLMs.
Layered architectures have been widely used in robot systems. The majority of them implement planning and execution functions in separate layers. However, there still lacks a straightforward way to transit high-level tasks in the planning layer to the low-level motor commands in the execution layer. In order to tackle this challenge, we propose a novel approach to ground the manipulator primitive tasks to robot low-level actions using large language models (LLMs). We designed a program-function-like prompt based on the task frame formalism. In this way, we enable LLMs to generate position/force set-points for hybrid control. Evaluations over several state-of-the-art LLMs are provided.