Fully Automated Task Management for Generation, Execution, and Evaluation: A Framework for Fetch-and-Carry Tasks with Natural Language Instructions in Continuous Space
This addresses the limitation of existing frameworks that rely on manual instructions and discrete actions, offering a more automated approach for robotics in continuous space.
The paper tackles the problem of enabling robots to execute fetch-and-carry tasks based on visual information and natural language instructions, proposing a framework for full automation of task generation, execution, and evaluation, with results including a division into four subtasks.
This paper aims to develop a framework that enables a robot to execute tasks based on visual information, in response to natural language instructions for Fetch-and-Carry with Object Grounding (FCOG) tasks. Although there have been many frameworks, they usually rely on manually given instruction sentences. Therefore, evaluations have only been conducted with fixed tasks. Furthermore, many multimodal language understanding models for the benchmarks only consider discrete actions. To address the limitations, we propose a framework for the full automation of the generation, execution, and evaluation of FCOG tasks. In addition, we introduce an approach to solving the FCOG tasks by dividing them into four distinct subtasks.