Automating Manual Tasks through Intuitive Robot Programming and Cognitive Robotics
This addresses the challenge of making robot programming accessible to non-experts, though it appears incremental as it builds on existing LLM and CV technologies.
The paper tackles the problem of enabling end-users to program robots for manual tasks by introducing an intuitive system that translates natural language and gestures into robot programs using LLMs and computer vision, with feedback mechanisms to ensure safety and user acceptance.
This paper presents a novel concept for intuitive end-user programming of robots, inspired by natural interaction between humans. Natural language and supportive gestures are translated into robot programs using large language models (LLMs) and computer vision (CV). Through equally natural system feedback in the form of clarification questions and visual representations, the generated program can be reviewed and adjusted, thereby ensuring safety, transparency, and user acceptance.