Quantum policy gradient algorithms
This work addresses the problem of limited applicability of quantum algorithms in reinforcement learning for researchers and practitioners in quantum computing and AI, offering incremental improvements with specific conditions.
The authors tackled the challenge of applying quantum algorithms to reinforcement learning in large state and action spaces by designing quantum policy gradient algorithms that exploit quantum interactions with environments, achieving quadratic speed-ups in sample complexity under certain regularity conditions, particularly for policies derived from parametrized quantum circuits.
Understanding the power and limitations of quantum access to data in machine learning tasks is primordial to assess the potential of quantum computing in artificial intelligence. Previous works have already shown that speed-ups in learning are possible when given quantum access to reinforcement learning environments. Yet, the applicability of quantum algorithms in this setting remains very limited, notably in environments with large state and action spaces. In this work, we design quantum algorithms to train state-of-the-art reinforcement learning policies by exploiting quantum interactions with an environment. However, these algorithms only offer full quadratic speed-ups in sample complexity over their classical analogs when the trained policies satisfy some regularity conditions. Interestingly, we find that reinforcement learning policies derived from parametrized quantum circuits are well-behaved with respect to these conditions, which showcases the benefit of a fully-quantum reinforcement learning framework.