When Object-Centric World Models Meet Policy Learning: From Pixels to Policies, and Where It Breaks
This addresses the challenge of stable control in reinforcement learning for researchers, but it is incremental as it identifies a limitation in existing object-centric approaches.
The paper tackled the problem of using object-centric world models to improve policy learning in reinforcement learning, finding that while their model DLPWM achieved strong visual reconstruction and robustness, policies trained on its latents underperformed compared to DreamerV3 due to representation shift during interactions.
Object-centric world models (OCWM) aim to decompose visual scenes into object-level representations, providing structured abstractions that could improve compositional generalization and data efficiency in reinforcement learning. We hypothesize that explicitly disentangled object-level representations, by localizing task-relevant information, can enhance policy performance across novel feature combinations. To test this hypothesis, we introduce DLPWM, a fully unsupervised, disentangled object-centric world model that learns object-level latents directly from pixels. DLPWM achieves strong reconstruction and prediction performance, including robustness to several out-of-distribution (OOD) visual variations. However, when used for downstream model-based control, policies trained on DLPWM latents underperform compared to DreamerV3. Through latent-trajectory analyses, we identify representation shift during multi-object interactions as a key driver of unstable policy learning. Our results suggest that, although object-centric perception supports robust visual modeling, achieving stable control requires mitigating latent drift.