Dynamics-aware Embeddings
This addresses the problem of high sample complexity in reinforcement learning for researchers and practitioners, offering an incremental improvement in sample efficiency.
The paper tackles improving sample efficiency in reinforcement learning by proposing a forward prediction objective to learn embeddings of states and action sequences that capture environment dynamics, resulting in efficient policy learning with 1-2 million environment steps for goal-conditioned continuous control from pixels.
In this paper we consider self-supervised representation learning to improve sample efficiency in reinforcement learning (RL). We propose a forward prediction objective for simultaneously learning embeddings of states and action sequences. These embeddings capture the structure of the environment's dynamics, enabling efficient policy learning. We demonstrate that our action embeddings alone improve the sample efficiency and peak performance of model-free RL on control from low-dimensional states. By combining state and action embeddings, we achieve efficient learning of high-quality policies on goal-conditioned continuous control from pixel observations in only 1-2 million environment steps.