Adversarial Inception Backdoor Attacks against Reinforcement Learning
This addresses a security vulnerability in DRL systems by enabling robust backdoor attacks that work under realistic conditions, which is incremental as it builds on prior work but introduces a new class of attacks.
The authors tackled the problem of backdoor poisoning attacks in Deep Reinforcement Learning (DRL) by proposing 'inception' attacks that manipulate training data to induce adversarial behavior under strict reward constraints, achieving a 100% attack success rate on multiple environments with minimal impact on task performance.
Recent works have demonstrated the vulnerability of Deep Reinforcement Learning (DRL) algorithms against training-time, backdoor poisoning attacks. The objectives of these attacks are twofold: induce pre-determined, adversarial behavior in the agent upon observing a fixed trigger during deployment while allowing the agent to solve its intended task during training. Prior attacks assume arbitrary control over the agent's rewards, inducing values far outside the environment's natural constraints. This results in brittle attacks that fail once the proper reward constraints are enforced. Thus, in this work we propose a new class of backdoor attacks against DRL which are the first to achieve state of the art performance under strict reward constraints. These "inception" attacks manipulate the agent's training data -- inserting the trigger into prior observations and replacing high return actions with those of the targeted adversarial behavior. We formally define these attacks and prove they achieve both adversarial objectives against arbitrary Markov Decision Processes (MDP). Using this framework we devise an online inception attack which achieves an 100\% attack success rate on multiple environments under constrained rewards while minimally impacting the agent's task performance.