World Models as Group Actions
For researchers building action-conditioned world models, this work provides a principled framework and practical method to ensure dynamics are truly governed by actions, addressing a key limitation in current video prediction models.
The paper argues that action faithfulness in video world models should be evaluated through the compositional group structure of actions (e.g., SE(2) for navigation). It proposes a regularization method enforcing group action properties (identity, inverse, composition) via latent-space supervision, and introduces metrics GAC and GAR, achieving consistent improvements in structural correctness and rollout stability without sacrificing visual quality.
Video world models have achieved strong visual realism, but this does not ensure that their dynamics are truly governed by actions. In this work, we argue that action faithfulness should be understood through the compositional structure of actions, which in many embodied settings follows a group structure (e.g., SE(2) for navigation). Based on this insight, we formalize action-conditioned world modeling as realizing a group action on the state space, providing a principled criterion for evaluating dynamics beyond visual quality. To operationalize this framework, we propose a unified approach that enforces identity, inverse, and composition consistency via latent-space regularization with synthesized supervision, avoiding additional data collection. We further introduce two metrics: Group-Action Consistency (GAC) and Group-Action Robustness (GAR), to evaluate structural correctness and rollout stability. Extensive experimental results show that our method consistently improves both GAC and GAR in state-of-the-art video world models without degrading perceptual quality.