Toward Accurate Long-Horizon Robotic Manipulation: Language-to-Action with Foundation Models via Scene Graphs
This work addresses the challenge of robotic manipulation for tasks requiring long-term planning, though it appears incremental as it integrates existing models rather than introducing a fundamentally new approach.
The paper tackles the problem of enabling accurate long-horizon robotic manipulation by developing a framework that uses pre-trained foundation models without domain-specific training, achieving results that demonstrate its potential for building robotic systems directly on these models.
This paper presents a framework that leverages pre-trained foundation models for robotic manipulation without domain-specific training. The framework integrates off-the-shelf models, combining multimodal perception from foundation models with a general-purpose reasoning model capable of robust task sequencing. Scene graphs, dynamically maintained within the framework, provide spatial awareness and enable consistent reasoning about the environment. The framework is evaluated through a series of tabletop robotic manipulation experiments, and the results highlight its potential for building robotic manipulation systems directly on top of off-the-shelf foundation models.