Stealthy Adversarial Attacks on Stochastic Multi-Armed Bandits
This work addresses security risks for users of multi-armed bandit algorithms in decision-making systems, but it is incremental as it builds on existing attack studies by introducing stealthiness.
The paper tackles the problem of adversarial attacks on stochastic multi-armed bandit algorithms by focusing on stealthy reward poisoning attacks that evade detection, finding that such attacks can succeed depending on environmental conditions and initial rewards, and revealing that stealthy attacks can almost always succeed against general algorithms with their detection method.
Adversarial attacks against stochastic multi-armed bandit (MAB) algorithms have been extensively studied in the literature. In this work, we focus on reward poisoning attacks and find most existing attacks can be easily detected by our proposed detection method based on the test of homogeneity, due to their aggressive nature in reward manipulations. This motivates us to study the notion of stealthy attack against stochastic MABs and investigate the resulting attackability. Our analysis shows that against two popularly employed MAB algorithms, UCB1 and $ε$-greedy, the success of a stealthy attack depends on the environmental conditions and the realized reward of the arm pulled in the first round. We also analyze the situation for general MAB algorithms equipped with our attack detection method and find that it is possible to have a stealthy attack that almost always succeeds. This brings new insights into the security risks of MAB algorithms.