Leveraging Language for Accelerated Learning of Tool Manipulation
This addresses the challenge of robust tool use in robotics, offering a domain-specific improvement for faster policy adaptation.
The paper tackled the problem of accelerating tool manipulation learning by using linguistic information about tools, and found that combining language features with meta-learning significantly speeds up adaptation to new tools in tasks like pushing and hammering.
Robust and generalized tool manipulation requires an understanding of the properties and affordances of different tools. We investigate whether linguistic information about a tool (e.g., its geometry, common uses) can help control policies adapt faster to new tools for a given task. We obtain diverse descriptions of various tools in natural language and use pre-trained language models to generate their feature representations. We then perform language-conditioned meta-learning to learn policies that can efficiently adapt to new tools given their corresponding text descriptions. Our results demonstrate that combining linguistic information and meta-learning significantly accelerates tool learning in several manipulation tasks including pushing, lifting, sweeping, and hammering.