Expert Selection in High-Dimensional Markov Decision Processes
This addresses the need for efficient expert policy selection at run-time in applications with multiple available policies, though it appears incremental as it builds on existing bandit algorithms.
The paper tackles the problem of online expert selection in high-dimensional Markov decision processes by introducing a multi-armed bandit framework that switches between candidate expert policies to identify the best one, achieving low regret in system performance.
In this work we present a multi-armed bandit framework for online expert selection in Markov decision processes and demonstrate its use in high-dimensional settings. Our method takes a set of candidate expert policies and switches between them to rapidly identify the best performing expert using a variant of the classical upper confidence bound algorithm, thus ensuring low regret in the overall performance of the system. This is useful in applications where several expert policies may be available, and one needs to be selected at run-time for the underlying environment.