Training Adversarial Agents to Exploit Weaknesses in Deep Control Policies
This work addresses safety-critical applications like autonomous vehicles by providing a method to identify weaknesses in deep learning-based control policies, though it is incremental as it builds on existing adversarial testing techniques.
The paper tackles the problem of validating safety in deep control policies by proposing an automated black box testing framework using adversarial reinforcement learning to exploit weaknesses, and demonstrates its effectiveness by finding hidden vulnerabilities in autonomous driving neural networks that manual testing missed.
Deep learning has become an increasingly common technique for various control problems, such as robotic arm manipulation, robot navigation, and autonomous vehicles. However, the downside of using deep neural networks to learn control policies is their opaque nature and the difficulties of validating their safety. As the networks used to obtain state-of-the-art results become increasingly deep and complex, the rules they have learned and how they operate become more challenging to understand. This presents an issue, since in safety-critical applications the safety of the control policy must be ensured to a high confidence level. In this paper, we propose an automated black box testing framework based on adversarial reinforcement learning. The technique uses an adversarial agent, whose goal is to degrade the performance of the target model under test. We test the approach on an autonomous vehicle problem, by training an adversarial reinforcement learning agent, which aims to cause a deep neural network-driven autonomous vehicle to collide. Two neural networks trained for autonomous driving are compared, and the results from the testing are used to compare the robustness of their learned control policies. We show that the proposed framework is able to find weaknesses in both control policies that were not evident during online testing and therefore, demonstrate a significant benefit over manual testing methods.