Federated Learning for Heterogeneous Bandits with Unobserved Contexts
This work addresses federated learning for bandit problems with noisy contexts, which is incremental as it extends existing methods to handle unobserved contexts in a distributed setting.
The paper tackles federated contextual bandits with unobserved contexts, where agents collaborate to minimize regret by sharing estimates with a central server, and proposes an elimination-based algorithm with a proven regret bound for linear reward functions, validated on synthetic and MovieLens data.
We study the problem of federated stochastic multi-arm contextual bandits with unknown contexts, in which M agents are faced with different bandits and collaborate to learn. The communication model consists of a central server and the agents share their estimates with the central server periodically to learn to choose optimal actions in order to minimize the total regret. We assume that the exact contexts are not observable and the agents observe only a distribution of the contexts. Such a situation arises, for instance, when the context itself is a noisy measurement or based on a prediction mechanism. Our goal is to develop a distributed and federated algorithm that facilitates collaborative learning among the agents to select a sequence of optimal actions so as to maximize the cumulative reward. By performing a feature vector transformation, we propose an elimination-based algorithm and prove the regret bound for linearly parametrized reward functions. Finally, we validated the performance of our algorithm and compared it with another baseline approach using numerical simulations on synthetic data and on the real-world movielens dataset.