Representing Positional Information in Generative World Models for Object Manipulation
This work addresses a key limitation in robotics for embodied agents by enhancing object manipulation capabilities, though it appears incremental as it builds on existing world model frameworks.
The paper tackles the problem of accurately manipulating objects in robotics by addressing how world models represent positional information, and introduces two methods (PCP and LCP) that improve performance in object-positioning tasks compared to current model-based approaches.
Object manipulation capabilities are essential skills that set apart embodied agents engaging with the world, especially in the realm of robotics. The ability to predict outcomes of interactions with objects is paramount in this setting. While model-based control methods have started to be employed for tackling manipulation tasks, they have faced challenges in accurately manipulating objects. As we analyze the causes of this limitation, we identify the cause of underperformance in the way current world models represent crucial positional information, especially about the target's goal specification for object positioning tasks. We introduce a general approach that empowers world model-based agents to effectively solve object-positioning tasks. We propose two declinations of this approach for generative world models: position-conditioned (PCP) and latent-conditioned (LCP) policy learning. In particular, LCP employs object-centric latent representations that explicitly capture object positional information for goal specification. This naturally leads to the emergence of multimodal capabilities, enabling the specification of goals through spatial coordinates or a visual goal. Our methods are rigorously evaluated across several manipulation environments, showing favorable performance compared to current model-based control approaches.