Reverse Experience Replay
This is an incremental improvement for reinforcement learning practitioners dealing with sparse reward environments.
The paper tackles the problem of sparse rewards in Deep Q-learning by introducing Reverse Experience Replay (RER), which samples transitions in reverse order, achieving competitive results against Double DQN and vanilla DQN in tasks with sufficient experience and memory, and significantly increased results in tasks with limited experience and memory.
This paper describes an improvement in Deep Q-learning called Reverse Experience Replay (also RER) that solves the problem of sparse rewards and helps to deal with reward maximizing tasks by sampling transitions successively in reverse order. On tasks with enough experience for training and enough Experience Replay memory capacity, Deep Q-learning Network with Reverse Experience Replay shows competitive results against both Double DQN, with a standard Experience Replay, and vanilla DQN. Also, RER achieves significantly increased results in tasks with a lack of experience and Replay memory capacity.