Learning Time-Sensitive Strategies in Space Fortress
This addresses challenges in reinforcement learning for complex, dynamic environments, though it appears incremental as it builds on existing algorithms.
The paper tackled the problem of reinforcement learning in games with reward sparsity, abrupt strategy reversals, and time-sensitive play, showing that state-of-the-art deep RL algorithms fail on these, and presented enhancements that led to big performance increases.
Although there has been remarkable progress and impressive performance on reinforcement learning (RL) on Atari games, there are many problems with challenging characteristics that have not yet been explored in Deep Learning for RL. These include reward sparsity, abrupt context-dependent reversals of strategy and time-sensitive game play. In this paper, we present Space Fortress, a game that incorporates all these characteristics and experimentally show that the presence of any of these renders state of the art Deep RL algorithms incapable of learning. Then, we present our enhancements to an existing algorithm and show big performance increases through each enhancement through an ablation study. We discuss how each of these enhancements was able to help and also argue that appropriate transfer learning boosts performance.