Practical Calculation of Gittins Indices for Multi-armed Bandits
This work addresses a practical obstacle for researchers and practitioners in reinforcement learning and decision-making, though it is incremental as it focuses on computational accessibility rather than new theoretical insights.
The paper tackled the problem of computing Gittins indices for multi-armed bandits, which are optimal but perceived as difficult to calculate, by providing an accessible methodology and open-source software for Bernoulli and Gaussian rewards, effectively removing computation as a barrier.
Gittins indices provide an optimal solution to the classical multi-armed bandit problem. An obstacle to their use has been the common perception that their computation is very difficult. This paper demonstrates an accessible general methodology for the calculating Gittins indices for the multi-armed bandit with a detailed study on the cases of Bernoulli and Gaussian rewards. With accompanying easy-to-use open source software, this work removes computation as a barrier to using Gittins indices in these commonly found settings.