Adaptive Clustering and Personalization in Multi-Agent Stochastic Linear Bandits
This work addresses efficient learning in multi-agent systems with similar but non-identical users, offering adaptive solutions for applications like recommendation systems, though it is incremental in extending bandit theory to heterogeneous settings.
The paper tackles regret minimization in multi-agent stochastic linear bandits with user heterogeneity, proposing adaptive algorithms for clustering and personalization frameworks that achieve regret bounds scaling with cluster separation or deviation from population average, such as O(√(T/N)) for well-separated clusters.
We consider the problem of minimizing regret in an $N$ agent heterogeneous stochastic linear bandits framework, where the agents (users) are similar but not all identical. We model user heterogeneity using two popularly used ideas in practice; (i) A clustering framework where users are partitioned into groups with users in the same group being identical to each other, but different across groups, and (ii) a personalization framework where no two users are necessarily identical, but a user's parameters are close to that of the population average. In the clustered users' setup, we propose a novel algorithm, based on successive refinement of cluster identities and regret minimization. We show that, for any agent, the regret scales as $\mathcal{O}(\sqrt{T/N})$, if the agent is in a `well separated' cluster, or scales as $\mathcal{O}(T^{\frac{1}{2} + \varepsilon}/(N)^{\frac{1}{2} -\varepsilon})$ if its cluster is not well separated, where $\varepsilon$ is positive and arbitrarily close to $0$. Our algorithm is adaptive to the cluster separation, and is parameter free -- it does not need to know the number of clusters, separation and cluster size, yet the regret guarantee adapts to the inherent complexity. In the personalization framework, we introduce a natural algorithm where, the personal bandit instances are initialized with the estimates of the global average model. We show that, an agent $i$ whose parameter deviates from the population average by $ε_i$, attains a regret scaling of $\widetilde{O}(ε_i\sqrt{T})$. This demonstrates that if the user representations are close (small $ε_i)$, the resulting regret is low, and vice-versa. The results are empirically validated and we observe superior performance of our adaptive algorithms over non-adaptive baselines.