Multi-Agent Bandit Learning through Heterogeneous Action Erasure Channels
This addresses communication constraints and delays in multi-agent distributed environments, offering a solution for applications like collaborative learning, but it is incremental as it builds on existing bandit algorithms with a new channel model.
The paper tackles the problem of multi-agent bandit learning with heterogeneous action erasure channels, where communication issues cause agents to miss actions, and introduces novel algorithms that achieve sub-linear regret, unlike existing methods with linear regret, as demonstrated through numerical experiments.
Multi-Armed Bandit (MAB) systems are witnessing an upswing in applications within multi-agent distributed environments, leading to the advancement of collaborative MAB algorithms. In such settings, communication between agents executing actions and the primary learner making decisions can hinder the learning process. A prevalent challenge in distributed learning is action erasure, often induced by communication delays and/or channel noise. This results in agents possibly not receiving the intended action from the learner, subsequently leading to misguided feedback. In this paper, we introduce novel algorithms that enable learners to interact concurrently with distributed agents across heterogeneous action erasure channels with different action erasure probabilities. We illustrate that, in contrast to existing bandit algorithms, which experience linear regret, our algorithms assure sub-linear regret guarantees. Our proposed solutions are founded on a meticulously crafted repetition protocol and scheduling of learning across heterogeneous channels. To our knowledge, these are the first algorithms capable of effectively learning through heterogeneous action erasure channels. We substantiate the superior performance of our algorithm through numerical experiments, emphasizing their practical significance in addressing issues related to communication constraints and delays in multi-agent environments.