Analysis and Improvement of Adversarial Training in DQN Agents With Adversarially-Guided Exploration (AGE)
This work addresses robustness in reinforcement learning agents for applications like robotics or gaming, but it appears incremental as it builds on existing adversarial training and exploration techniques.
This paper tackles the problem of improving the robustness of Deep Q-Network (DQN) policies to state-space perturbations by analyzing adversarial training and proposing a novel Adversarially-Guided Exploration (AGE) mechanism to address sample inefficiency, with experimental verification showing performance comparisons against traditional methods.
This paper investigates the effectiveness of adversarial training in enhancing the robustness of Deep Q-Network (DQN) policies to state-space perturbations. We first present a formal analysis of adversarial training in DQN agents and its performance with respect to the proportion of adversarial perturbations to nominal observations used for training. Next, we consider the sample-inefficiency of current adversarial training techniques, and propose a novel Adversarially-Guided Exploration (AGE) mechanism based on a modified hybrid of the $ε$-greedy algorithm and Boltzmann exploration. We verify the feasibility of this exploration mechanism through experimental evaluation of its performance in comparison with the traditional decaying $ε$-greedy and parameter-space noise exploration algorithms.