Tackling Visual Control via Multi-View Exploration Maximization
This addresses the problem of sample inefficiency and poor generalization in reinforcement learning for real-world applications with high-dimensional observations, representing an incremental improvement over existing methods.
The paper tackles complex visual control tasks by proposing MEM, a method that combines multi-view representation learning with intrinsic reward-driven exploration in reinforcement learning, achieving superior performance and higher efficiency on tasks from DeepMind Control Suite and Procgen games.
We present MEM: Multi-view Exploration Maximization for tackling complex visual control tasks. To the best of our knowledge, MEM is the first approach that combines multi-view representation learning and intrinsic reward-driven exploration in reinforcement learning (RL). More specifically, MEM first extracts the specific and shared information of multi-view observations to form high-quality features before performing RL on the learned features, enabling the agent to fully comprehend the environment and yield better actions. Furthermore, MEM transforms the multi-view features into intrinsic rewards based on entropy maximization to encourage exploration. As a result, MEM can significantly promote the sample-efficiency and generalization ability of the RL agent, facilitating solving real-world problems with high-dimensional observations and spare-reward space. We evaluate MEM on various tasks from DeepMind Control Suite and Procgen games. Extensive simulation results demonstrate that MEM can achieve superior performance and outperform the benchmarking schemes with simple architecture and higher efficiency.