Learning Massively Multitask World Models for Continuous Control
This work addresses the problem of general-purpose control for AI agents across many tasks and embodiments, representing a novel method for a known bottleneck rather than an incremental improvement.
The authors tackled the challenge of scaling online reinforcement learning for continuous control across diverse tasks and embodiments by introducing a new benchmark with 200 tasks and presenting Newt, a language-conditioned multitask world model. Newt achieved better multitask performance and data-efficiency than strong baselines, exhibited strong open-loop control, and enabled rapid adaptation to unseen tasks.
General-purpose control demands agents that act across many tasks and embodiments, yet research on reinforcement learning (RL) for continuous control remains dominated by single-task or offline regimes, reinforcing a view that online RL does not scale. Inspired by the foundation model recipe (large-scale pretraining followed by light RL) we ask whether a single agent can be trained on hundreds of tasks with online interaction. To accelerate research in this direction, we introduce a new benchmark with 200 diverse tasks spanning many domains and embodiments, each with language instructions, demonstrations, and optionally image observations. We then present \emph{Newt}, a language-conditioned multitask world model that is first pretrained on demonstrations to acquire task-aware representations and action priors, and then jointly optimized with online interaction across all tasks. Experiments show that Newt yields better multitask performance and data-efficiency than a set of strong baselines, exhibits strong open-loop control, and enables rapid adaptation to unseen tasks. We release our environments, demonstrations, code for training and evaluation, as well as 200+ checkpoints.