Unsupervised Control Through Non-Parametric Discriminative Rewards
This addresses the challenge of unsupervised reinforcement learning for researchers, offering a method to learn goal-conditioned policies and reward functions without external supervision, though it appears incremental as it builds on existing unsupervised and goal-conditioned approaches.
The paper tackles the problem of learning to control environments without hand-crafted rewards or expert data by presenting an unsupervised algorithm that trains agents to achieve perceptually-specified goals using only observations and actions, demonstrating efficacy in reaching diverse goals on Atari, DeepMind Control Suite, and DeepMind Lab domains.
Learning to control an environment without hand-crafted rewards or expert data remains challenging and is at the frontier of reinforcement learning research. We present an unsupervised learning algorithm to train agents to achieve perceptually-specified goals using only a stream of observations and actions. Our agent simultaneously learns a goal-conditioned policy and a goal achievement reward function that measures how similar a state is to the goal state. This dual optimization leads to a co-operative game, giving rise to a learned reward function that reflects similarity in controllable aspects of the environment instead of distance in the space of observations. We demonstrate the efficacy of our agent to learn, in an unsupervised manner, to reach a diverse set of goals on three domains -- Atari, the DeepMind Control Suite and DeepMind Lab.