Full Gradient DQN Reinforcement Learning: A Provably Convergent Scheme
This work addresses theoretical convergence problems in DQN for reinforcement learning researchers, but it appears incremental as it modifies an existing algorithm without broad SOTA impact.
The paper tackled theoretical issues in DQN reinforcement learning by proposing Full Gradient DQN (FG-DQN), a modified scheme with a sound theoretical basis, and observed better performance compared to the original DQN on sample problems, though no concrete numbers were provided.
We analyze the DQN reinforcement learning algorithm as a stochastic approximation scheme using the o.d.e. (for 'ordinary differential equation') approach and point out certain theoretical issues. We then propose a modified scheme called Full Gradient DQN (FG-DQN, for short) that has a sound theoretical basis and compare it with the original scheme on sample problems. We observe a better performance for FG-DQN.