LIMT: Language-Informed Multi-Task Visual World Models
This work addresses the problem of enabling robots to perform multiple tasks efficiently, which is incremental as it builds on existing model-based and language-driven techniques.
The paper tackles the challenge of multi-task reinforcement learning for robots by proposing a model-based approach that uses pre-trained language models to extract task representations, resulting in improved sample efficiency and performance over model-free methods.
Most recent successes in robot reinforcement learning involve learning a specialized single-task agent. However, robots capable of performing multiple tasks can be much more valuable in real-world applications. Multi-task reinforcement learning can be very challenging due to the increased sample complexity and the potentially conflicting task objectives. Previous work on this topic is dominated by model-free approaches. The latter can be very sample inefficient even when learning specialized single-task agents. In this work, we focus on model-based multi-task reinforcement learning. We propose a method for learning multi-task visual world models, leveraging pre-trained language models to extract semantically meaningful task representations. These representations are used by the world model and policy to reason about task similarity in dynamics and behavior. Our results highlight the benefits of using language-driven task representations for world models and a clear advantage of model-based multi-task learning over the more common model-free paradigm.