Model-Based Regularization for Deep Reinforcement Learning with Transcoder Networks
This work addresses sample efficiency and performance issues in deep reinforcement learning for Atari game benchmarks, representing an incremental improvement.
The paper tackled the problem of improving sample efficiency and performance in deep reinforcement learning by proposing a model-based regularization objective for Deep Q-Networks, achieving superior results over vanilla DQN on 20 Atari games.
This paper proposes a new optimization objective for value-based deep reinforcement learning. We extend conventional Deep Q-Networks (DQNs) by adding a model-learning component yielding a transcoder network. The prediction errors for the model are included in the basic DQN loss as additional regularizers. This augmented objective leads to a richer training signal that provides feedback at every time step. Moreover, because learning an environment model shares a common structure with the RL problem, we hypothesize that the resulting objective improves both sample efficiency and performance. We empirically confirm our hypothesis on a range of 20 games from the Atari benchmark attaining superior results over vanilla DQN without model-based regularization.