Environment Agnostic Goal-Conditioning, A Study of Reward-Free Autonomous Learning
This work addresses the challenge of reward-free autonomous learning for AI agents, though it is incremental as it builds on existing goal-conditioned reinforcement learning methods.
The paper tackles the problem of enabling agents to learn tasks autonomously without external rewards by transforming environments into goal-conditioned settings, showing that agents can achieve training times comparable to externally guided reinforcement learning with improved average goal success rates.
In this paper we study how transforming regular reinforcement learning environments into goal-conditioned environments can let agents learn to solve tasks autonomously and reward-free. We show that an agent can learn to solve tasks by selecting its own goals in an environment-agnostic way, at training times comparable to externally guided reinforcement learning. Our method is independent of the underlying off-policy learning algorithm. Since our method is environment-agnostic, the agent does not value any goals higher than others, leading to instability in performance for individual goals. However, in our experiments, we show that the average goal success rate improves and stabilizes. An agent trained with this method can be instructed to seek any observations made in the environment, enabling generic training of agents prior to specific use cases.