General agents contain world models
This provides a foundational insight for developing safe and general AI agents, with implications for bounding capabilities and designing new algorithms.
The paper tackles the question of whether world models are necessary for flexible, goal-directed behavior by proving that any agent generalizing to multi-step tasks must have learned a predictive environment model, which can be extracted from its policy and improves with performance and goal complexity.
Are world models a necessary ingredient for flexible, goal-directed behaviour, or is model-free learning sufficient? We provide a formal answer to this question, showing that any agent capable of generalizing to multi-step goal-directed tasks must have learned a predictive model of its environment. We show that this model can be extracted from the agent's policy, and that increasing the agents performance or the complexity of the goals it can achieve requires learning increasingly accurate world models. This has a number of consequences: from developing safe and general agents, to bounding agent capabilities in complex environments, and providing new algorithms for eliciting world models from agents.