Robust Decentralized Multi-armed Bandits: From Corruption-Resilience to Byzantine-Resilience
This addresses security concerns in decentralized collaborative learning systems where agents may face adversarial manipulation, representing a significant but incremental advance over existing corruption-susceptible methods.
The paper tackles the vulnerability of decentralized multi-agent multi-armed bandits to adversarial attacks by proposing DeMABAR, a robust algorithm that limits individual regret to an additive term proportional to the corruption budget in corruption-resilient settings and nearly eliminates attack influence in Byzantine settings.
Decentralized cooperative multi-agent multi-armed bandits (DeCMA2B) considers how multiple agents collaborate in a decentralized multi-armed bandit setting. Though this problem has been extensively studied in previous work, most existing methods remain susceptible to various adversarial attacks. In this paper, we first study DeCMA2B with adversarial corruption, where an adversary can corrupt reward observations of all agents with a limited corruption budget. We propose a robust algorithm, called DeMABAR, which ensures that each agent's individual regret suffers only an additive term proportional to the corruption budget. Then we consider a more realistic scenario where the adversary can only attack a small number of agents. Our theoretical analysis shows that the DeMABAR algorithm can also almost completely eliminate the influence of adversarial attacks and is inherently robust in the Byzantine setting, where an unknown fraction of the agents can be Byzantine, i.e., may arbitrarily select arms and communicate wrong information. We also conduct numerical experiments to illustrate the robustness and effectiveness of the proposed method.