Geometry-Aware Approaches for Balancing Performance and Theoretical Guarantees in Linear Bandits
This addresses a foundational problem in bandit algorithms for researchers and practitioners by bridging theory and practice, though it is incremental as it builds on existing methods.
The paper tackles the discrepancy between empirical performance and theoretical regret bounds in linear bandits by proposing a data-driven technique that tracks geometric properties of uncertainty ellipsoids, achieving minimax optimal regret of order ˜O(d√T) while preserving empirical efficacy.
This paper is motivated by recent research in the $d$-dimensional stochastic linear bandit literature, which has revealed an unsettling discrepancy: algorithms like Thompson sampling and Greedy demonstrate promising empirical performance, yet this contrasts with their pessimistic theoretical regret bounds. The challenge arises from the fact that while these algorithms may perform poorly in certain problem instances, they generally excel in typical instances. To address this, we propose a new data-driven technique that tracks the geometric properties of the uncertainty ellipsoid around the main problem parameter. This methodology enables us to formulate a data-driven frequentist regret bound, which incorporates the geometric information, for a broad class of base algorithms, including Greedy, OFUL, and Thompson sampling. This result allows us to identify and ``course-correct" problem instances in which the base algorithms perform poorly. The course-corrected algorithms achieve the minimax optimal regret of order $\tilde{\mathcal{O}}(d\sqrt{T})$ for a $T$-period decision-making scenario, effectively maintaining the desirable attributes of the base algorithms, including their empirical efficacy. We present simulation results to validate our findings using synthetic and real data.