Leveraging Initial Hints for Free in Stochastic Linear Bandits
This work addresses the challenge of efficiently incorporating prior knowledge in bandit algorithms for decision-making under uncertainty, offering a novel adaptive approach that is not incremental.
The paper tackles the problem of optimizing with bandit feedback by using an initial hint of the optimal action to improve regret bounds in stochastic linear bandits, achieving $ ilde O(\sqrt{T})$ regret when the hint is accurate while maintaining minimax-optimal $ ilde O(d\sqrt{T})$ regret regardless of hint quality, with extensions to multiple hints yielding $ ilde O(m^{2/3}\sqrt{T})$ regret.
We study the setting of optimizing with bandit feedback with additional prior knowledge provided to the learner in the form of an initial hint of the optimal action. We present a novel algorithm for stochastic linear bandits that uses this hint to improve its regret to $\tilde O(\sqrt{T})$ when the hint is accurate, while maintaining a minimax-optimal $\tilde O(d\sqrt{T})$ regret independent of the quality of the hint. Furthermore, we provide a Pareto frontier of tight tradeoffs between best-case and worst-case regret, with matching lower bounds. Perhaps surprisingly, our work shows that leveraging a hint shows provable gains without sacrificing worst-case performance, implying that our algorithm adapts to the quality of the hint for free. We also provide an extension of our algorithm to the case of $m$ initial hints, showing that we can achieve a $\tilde O(m^{2/3}\sqrt{T})$ regret.