On Distributed Cooperative Decision-Making in Multiarmed Bandits
This work addresses distributed decision-making problems in multi-agent systems, but it appears incremental as it builds on existing UCB methods by adding cooperative elements.
The paper tackles the explore-exploit tradeoff in distributed cooperative multi-armed bandits by designing the cooperative UCB algorithm, which integrates consensus algorithms for reward estimation and UCB-based heuristics for arm selection, and analyzes how communication graph structure affects group decision-making performance.
We study the explore-exploit tradeoff in distributed cooperative decision-making using the context of the multiarmed bandit (MAB) problem. For the distributed cooperative MAB problem, we design the cooperative UCB algorithm that comprises two interleaved distributed processes: (i) running consensus algorithms for estimation of rewards, and (ii) upper-confidence-bound-based heuristics for selection of arms. We rigorously analyze the performance of the cooperative UCB algorithm and characterize the influence of communication graph structure on the decision-making performance of the group.