Reinforcement learning with learned gadgets to tackle hard quantum problems on real hardware
This work addresses the problem of efficiently optimizing quantum circuits for researchers and practitioners in quantum computing, offering a novel method for enhancing exploration in parameterized quantum circuits, though it is incremental as it builds on existing reinforcement learning and program synthesis techniques.
The paper tackles the challenge of designing quantum circuits for complex tasks like approximating ground states of quantum Hamiltonians, an NP-hard problem, by introducing gadget reinforcement learning (GRL), which integrates reinforcement learning with program synthesis to automatically generate and incorporate composite gates, resulting in improved accuracy, hardware compatibility, and scalability under typical computational budgets (e.g., 2-3 days of GPU runtime).
Designing quantum circuits for specific tasks is challenging due to the exponential growth of the state space. We introduce gadget reinforcement learning (GRL), which integrates reinforcement learning with program synthesis to automatically generate and incorporate composite gates (gadgets) into the action space. This enhances the exploration of parameterized quantum circuits (PQCs) for complex tasks like approximating ground states of quantum Hamiltonians, an NP-hard problem. We evaluate GRL using the transverse field Ising model under typical computational budgets (e.g., 2- 3 days of GPU runtime). Our results show improved accuracy, hardware compatibility and scalability. GRL exhibits robust performance as the size and complexity of the problem increases, even with constrained computational resources. By integrating gadget extraction, GRL facilitates the discovery of reusable circuit components tailored for specific hardware, bridging the gap between algorithmic design and practical implementation. This makes GRL a versatile framework for optimizing quantum circuits with applications in hardware-specific optimizations and variational quantum algorithms. The code is available at: https://github.com/Aqasch/Gadget_RL