Worst-Case Regret Bounds for Exploration via Randomized Value Functions
This addresses the exploration challenge in reinforcement learning for researchers and practitioners, but it is incremental as it builds on existing methods with a theoretical analysis.
The paper tackles the problem of exploration in reinforcement learning by using randomized value functions, and shows that planning with these functions leads to provably efficient exploration with a worst-case regret bound for tabular finite-horizon Markov decision processes.
This paper studies a recent proposal to use randomized value functions to drive exploration in reinforcement learning. These randomized value functions are generated by injecting random noise into the training data, making the approach compatible with many popular methods for estimating parameterized value functions. By providing a worst-case regret bound for tabular finite-horizon Markov decision processes, we show that planning with respect to these randomized value functions can induce provably efficient exploration.