Learning Robust Representations via Bidirectional Transition for Visual Reinforcement Learning
This addresses the problem of representation reliability for visual reinforcement learning practitioners, though it appears incremental as it builds on existing transition prediction concepts.
The paper tackles the challenge of extracting reliable and generalizable representations from vision-based observations in visual reinforcement learning by introducing a Bidirectional Transition (BiT) model that predicts environmental transitions both forward and backward. The model demonstrates competitive generalization performance and sample efficiency on the DeepMind Control suite and shows wide applicability in robotic manipulation and CARLA simulators.
Visual reinforcement learning has proven effective in solving control tasks with high-dimensional observations. However, extracting reliable and generalizable representations from vision-based observations remains a central challenge. Inspired by the human thought process, when the representation extracted from the observation can predict the future and trace history, the representation is reliable and accurate in comprehending the environment. Based on this concept, we introduce a Bidirectional Transition (BiT) model, which leverages the ability to bidirectionally predict environmental transitions both forward and backward to extract reliable representations. Our model demonstrates competitive generalization performance and sample efficiency on two settings of the DeepMind Control suite. Additionally, we utilize robotic manipulation and CARLA simulators to demonstrate the wide applicability of our method.