Quantum Speedups in Regret Analysis of Infinite Horizon Average-Reward Markov Decision Processes
It addresses the challenge of enhancing regret bounds in reinforcement learning for unknown MDPs, offering a potential quantum speedup that is foundational rather than incremental.
This paper tackles the problem of infinite horizon average-reward Markov Decision Processes (MDPs) by introducing a quantum framework that leverages quantum mean estimation, resulting in an exponential improvement in regret guarantees from $ ilde{\mathcal{O}}(\sqrt{T})$ classically to $ ilde{\mathcal{O}}(1)$.
This paper investigates the potential of quantum acceleration in addressing infinite horizon Markov Decision Processes (MDPs) to enhance average reward outcomes. We introduce an innovative quantum framework for the agent's engagement with an unknown MDP, extending the conventional interaction paradigm. Our approach involves the design of an optimism-driven tabular Reinforcement Learning algorithm that harnesses quantum signals acquired by the agent through efficient quantum mean estimation techniques. Through thorough theoretical analysis, we demonstrate that the quantum advantage in mean estimation leads to exponential advancements in regret guarantees for infinite horizon Reinforcement Learning. Specifically, the proposed Quantum algorithm achieves a regret bound of $\tilde{\mathcal{O}}(1)$, a significant improvement over the $\tilde{\mathcal{O}}(\sqrt{T})$ bound exhibited by classical counterparts.