Goal-oriented Trajectories for Efficient Exploration
This addresses the problem of inefficient exploration in reinforcement learning for agents in environments like video games, though it is incremental as it builds on existing curiosity-based methods.
The paper tackles the challenge of exploration in reinforcement learning by proposing a method that replaces random action choices with random goal selection followed by directed steps, which generates longer and safer trajectories. It demonstrates this approach in Super Mario Bros., significantly improving the score of a baseline DQN agent.
Exploration is a difficult challenge in reinforcement learning and even recent state-of-the art curiosity-based methods rely on the simple epsilon-greedy strategy to generate novelty. We argue that pure random walks do not succeed to properly expand the exploration area in most environments and propose to replace single random action choices by random goals selection followed by several steps in their direction. This approach is compatible with any curiosity-based exploration and off-policy reinforcement learning agents and generates longer and safer trajectories than individual random actions. To illustrate this, we present a task-independent agent that learns to reach coordinates in screen frames and demonstrate its ability to explore with the game Super Mario Bros. improving significantly the score of a baseline DQN agent.