What Does it Mean for a Neural Network to Learn a "World Model"?
This work addresses the need for a common operational definition in AI research to clarify informal terms, but it is incremental as it builds on existing linear probing ideas and leaves action modeling for future work.
The authors tackled the problem of defining what it means for a neural network to learn a 'world model' by proposing precise criteria based on linear probing, focusing on latent state space representation, and they provided conditions to ensure the model is non-trivial, without reporting concrete numerical results.
We propose a set of precise criteria for saying a neural net learns and uses a "world model." The goal is to give an operational meaning to terms that are often used informally, in order to provide a common language for experimental investigation. We focus specifically on the idea of representing a latent "state space" of the world, leaving modeling the effect of actions to future work. Our definition is based on ideas from the linear probing literature, and formalizes the notion of a computation that factors through a representation of the data generation process. An essential addition to the definition is a set of conditions to check that such a "world model" is not a trivial consequence of the neural net's data or task.