Context and Diversity Matter: The Emergence of In-Context Learning in World Models
This work addresses the challenge of adapting world models to dynamic environments for embodied AI, representing an incremental advance by shifting focus from zero-shot to in-context learning.
The paper tackles the problem of static world models failing with novel or rare environmental configurations by investigating in-context environment learning (ICEL), focusing on growth and asymptotic limits rather than zero-shot performance. It formalizes ICEL, derives error upper-bounds, and empirically confirms distinct mechanisms, highlighting the necessity of long context and diverse environments.
The capability of predicting environmental dynamics underpins both biological neural systems and general embodied AI in adapting to their surroundings. Yet prevailing approaches rest on static world models that falter when confronted with novel or rare configurations. We investigate in-context environment learning (ICEL), shifting attention from zero-shot performance to the growth and asymptotic limits of the world model. Our contributions are three-fold: (1) we formalize in-context learning of a world model and identify two core mechanisms: environment recognition and environment learning; (2) we derive error upper-bounds for both mechanisms that expose how the mechanisms emerge; and (3) we empirically confirm that distinct ICL mechanisms exist in the world model, and we further investigate how data distribution and model architecture affect ICL in a manner consistent with theory. These findings demonstrate the potential of self-adapting world models and highlight the key factors behind the emergence of ICEL, most notably the necessity of long context and diverse environments.