Nearly-tight Approximation Guarantees for the Improving Multi-Armed Bandits Problem
This addresses a theoretical problem in online learning for bandit algorithms with non-stationary rewards, offering tight approximation guarantees that are incremental over prior work.
The paper tackles the improving multi-armed bandits problem, where arms have concave increasing reward functions, and provides nearly-tight bounds by showing a lower bound of Ω(√k) and an algorithm achieving O(√k log k) approximation relative to optimal.
We give nearly-tight upper and lower bounds for the improving multi-armed bandits problem. An instance of this problem has $k$ arms, each of whose reward function is a concave and increasing function of the number of times that arm has been pulled so far. We show that for any randomized online algorithm, there exists an instance on which it must suffer at least an $Ω(\sqrt{k})$ approximation factor relative to the optimal reward. We then provide a randomized online algorithm that guarantees an $O(\sqrt{k})$ approximation factor, if it is told the maximum reward achievable by the optimal arm in advance. We then show how to remove this assumption at the cost of an extra $O(\log k)$ approximation factor, achieving an overall $O(\sqrt{k} \log k)$ approximation relative to optimal.