Defending Observation Attacks in Deep Reinforcement Learning via Detection and Denoising
This addresses the vulnerability of DRL policies to observation attacks in continuous control benchmarks, offering a safer training alternative, though it is incremental as it builds on existing defense concepts.
The paper tackles the problem of adversarial attacks on deep reinforcement learning policies by proposing a detect-and-denoise defense strategy, which achieves performance comparable to state-of-the-art adversarial training methods without requiring data sampling under attack.
Neural network policies trained using Deep Reinforcement Learning (DRL) are well-known to be susceptible to adversarial attacks. In this paper, we consider attacks manifesting as perturbations in the observation space managed by the external environment. These attacks have been shown to downgrade policy performance significantly. We focus our attention on well-trained deterministic and stochastic neural network policies in the context of continuous control benchmarks subject to four well-studied observation space adversarial attacks. To defend against these attacks, we propose a novel defense strategy using a detect-and-denoise schema. Unlike previous adversarial training approaches that sample data in adversarial scenarios, our solution does not require sampling data in an environment under attack, thereby greatly reducing risk during training. Detailed experimental results show that our technique is comparable with state-of-the-art adversarial training approaches.