From Classical Data to Quantum Advantage -- Quantum Policy Evaluation on Quantum Hardware
This work addresses the challenge of applying quantum reinforcement learning to real-world data for researchers in quantum computing and AI, though it is incremental as it builds on prior quantum simulation methods.
The paper tackles the problem of enabling quantum policy evaluation (QPE) on quantum hardware by learning environment parameters from classical data, demonstrating that this integration shows promising potential for quantum advantage in reinforcement learning despite hardware challenges.
Quantum policy evaluation (QPE) is a reinforcement learning (RL) algorithm which is quadratically more efficient than an analogous classical Monte Carlo estimation. It makes use of a direct quantum mechanical realization of a finite Markov decision process, in which the agent and the environment are modeled by unitary operators and exchange states, actions, and rewards in superposition. Previously, the quantum environment has been implemented and parametrized manually for an illustrative benchmark using a quantum simulator. In this paper, we demonstrate how these environment parameters can be learned from a batch of classical observational data through quantum machine learning (QML) on quantum hardware. The learned quantum environment is then applied in QPE to also compute policy evaluations on quantum hardware. Our experiments reveal that, despite challenges such as noise and short coherence times, the integration of QML and QPE shows promising potential for achieving quantum advantage in RL.