Uniform State Abstraction For Reinforcement Learning
This is an incremental improvement for reinforcement learning practitioners, addressing a specific bottleneck in applying abstract knowledge methods to deep learning.
The paper tackles the problem of MultiGrid Reinforcement Learning (MRL) not extending well to deep learning by improving it to work with algorithms like Deep Q-Networks (DQN), resulting in significantly better performance on continuous control tasks compared to vanilla DQN and DQN augmented with MRL.
Potential Based Reward Shaping combined with a potential function based on appropriately defined abstract knowledge has been shown to significantly improve learning speed in Reinforcement Learning. MultiGrid Reinforcement Learning (MRL) has further shown that such abstract knowledge in the form of a potential function can be learned almost solely from agent interaction with the environment. However, we show that MRL faces the problem of not extending well to work with Deep Learning. In this paper we extend and improve MRL to take advantage of modern Deep Learning algorithms such as Deep Q-Networks (DQN). We show that DQN augmented with our approach perform significantly better on continuous control tasks than its Vanilla counterpart and DQN augmented with MRL.