Relevance-Guided Modeling of Object Dynamics for Reinforcement Learning
This work addresses computational and sample efficiency issues in reinforcement learning for AI researchers, offering a novel approach that is incremental over prior unsupervised object representation methods.
The paper tackles the problem of inefficient deep reinforcement learning by proposing a framework for learning task-specific object representations through reasoning about object dynamics and interactions, demonstrating faster learning on Atari games compared to existing deep RL algorithms.
Current deep reinforcement learning (RL) approaches incorporate minimal prior knowledge about the environment, limiting computational and sample efficiency. \textit{Objects} provide a succinct and causal description of the world, and many recent works have proposed unsupervised object representation learning using priors and losses over static object properties like visual consistency. However, object dynamics and interactions are also critical cues for objectness. In this paper we propose a framework for reasoning about object dynamics and behavior to rapidly determine minimal and task-specific object representations. To demonstrate the need to reason over object behavior and dynamics, we introduce a suite of RGBD MuJoCo object collection and avoidance tasks that, while intuitive and visually simple, confound state-of-the-art unsupervised object representation learning algorithms. We also highlight the potential of this framework on several Atari games, using our object representation and standard RL and planning algorithms to learn dramatically faster than existing deep RL algorithms.