Photonic Quantum Policy Learning in OpenAI Gym
This work addresses the challenge of leveraging NISQ quantum devices for reinforcement learning, which is an emerging field, but it is incremental as it builds on existing quantum machine learning methods.
The authors tackled the problem of applying quantum reinforcement learning to classical continuous control tasks by introducing proximal policy optimization for photonic variational quantum agents, achieving comparable performance and faster convergence than a classical neural network baseline in the CartPole environment.
In recent years, near-term noisy intermediate scale quantum (NISQ) computing devices have become available. One of the most promising application areas to leverage such NISQ quantum computer prototypes is quantum machine learning. While quantum neural networks are widely studied for supervised learning, quantum reinforcement learning is still just an emerging field of this area. To solve a classical continuous control problem, we use a continuous-variable quantum machine learning approach. We introduce proximal policy optimization for photonic variational quantum agents and also study the effect of the data re-uploading. We present performance assessment via empirical study using Strawberry Fields, a photonic simulator Fock backend and a hybrid training framework connected to an OpenAI Gym environment and TensorFlow. For the restricted CartPole problem, the two variations of the photonic policy learning achieve comparable performance levels and a faster convergence than the baseline classical neural network of same number of trainable parameters.