Efficient RL via Disentangled Environment and Agent Representations
This addresses the problem of inefficient RL in visual domains for robotics and simulation applications, offering a novel approach with broad applicability.
The paper tackles the problem of improving reinforcement learning (RL) efficiency by learning disentangled representations of the environment and agent, using inexpensive visual knowledge like agent shape or mask. It shows that their method, Structured Environment-Agent Representations, outperforms state-of-the-art model-free approaches across 18 challenging visual simulation environments and 5 different robots.
Agents that are aware of the separation between themselves and their environments can leverage this understanding to form effective representations of visual input. We propose an approach for learning such structured representations for RL algorithms, using visual knowledge of the agent, such as its shape or mask, which is often inexpensive to obtain. This is incorporated into the RL objective using a simple auxiliary loss. We show that our method, Structured Environment-Agent Representations, outperforms state-of-the-art model-free approaches over 18 different challenging visual simulation environments spanning 5 different robots. Website at https://sear-rl.github.io/