Combinatorial Bandits for Maximum Value Reward Function under Max Value-Index Feedback
This work addresses a novel feedback scenario in bandit problems, which is incremental but provides theoretical guarantees for stochastic arm distributions.
The paper tackles a combinatorial multi-armed bandit problem with a new feedback structure between semi-bandit and full-bandit, proposing an algorithm that achieves O((k/Δ)log(T)) distribution-dependent and Õ(√T) distribution-independent regret bounds, comparable to more informative feedback settings.
We consider a combinatorial multi-armed bandit problem for maximum value reward function under maximum value and index feedback. This is a new feedback structure that lies in between commonly studied semi-bandit and full-bandit feedback structures. We propose an algorithm and provide a regret bound for problem instances with stochastic arm outcomes according to arbitrary distributions with finite supports. The regret analysis rests on considering an extended set of arms, associated with values and probabilities of arm outcomes, and applying a smoothness condition. Our algorithm achieves a $O((k/Δ)\log(T))$ distribution-dependent and a $\tilde{O}(\sqrt{T})$ distribution-independent regret where $k$ is the number of arms selected in each round, $Δ$ is a distribution-dependent reward gap and $T$ is the horizon time. Perhaps surprisingly, the regret bound is comparable to previously-known bound under more informative semi-bandit feedback. We demonstrate the effectiveness of our algorithm through experimental results.