Expert-augmented actor-critic for ViZDoom and Montezumas Revenge
This addresses sparse reward challenges in reinforcement learning for domains like video games, but appears incremental as it builds on existing actor-critic and expert-augmentation methods.
The paper tackles the problem of sparse rewards in reinforcement learning by proposing an expert-augmented actor-critic algorithm, achieving scores above 27,000 points on Montezuma's Revenge and surpassing expert data performance with proper hyperparameters.
We propose an expert-augmented actor-critic algorithm, which we evaluate on two environments with sparse rewards: Montezumas Revenge and a demanding maze from the ViZDoom suite. In the case of Montezumas Revenge, an agent trained with our method achieves very good results consistently scoring above 27,000 points (in many experiments beating the first world). With an appropriate choice of hyperparameters, our algorithm surpasses the performance of the expert data. In a number of experiments, we have observed an unreported bug in Montezumas Revenge which allowed the agent to score more than 800,000 points.