GeoSem-WAM: Geometry- and Semantic-Aware World Action Models
For embodied AI researchers, this work incrementally improves world action models by incorporating structured supervision, though the core idea of representation learning over future prediction is not new.
The paper investigates whether World Action Models benefit from explicit future imagination or representation learning, finding the latter more important. They propose GeoSem-WAM, which adds geometric and semantic prediction branches to improve latent representations, achieving better action prediction and robustness without test-time rollout.
Recent World Action Models (WAMs) have demonstrated impressive capabilities in embodied decision-making. However, whether their effectiveness stems from explicit future imagination during inference or representation learning induced by predictive training remains an open question. Emerging evidence suggests the primary advantage lies in learning robust latent representations rather than generating future observations at test time. Nevertheless, existing WAMs mainly rely on RGB-based future prediction, which provides limited structural and spatial understanding of complex environments. To address this, we propose a structured world modeling framework that enhances latent representations through geometric and semantic supervision. Alongside future RGB prediction, our model introduces two auxiliary prediction branches for future geometry and semantic representations, enabling it to jointly capture scene dynamics, spatial geometry, and semantic context within a unified latent space. Crucially, our approach preserves efficient inference by avoiding explicit future rollout or video generation at test time. Extensive experiments show that incorporating structured world supervision consistently improves action prediction accuracy, scene understanding, and robustness under challenging embodied scenarios, highlighting its potential for advancing scalable and efficient WAMs.