Vectorized Attention with Learnable Encoding for Quantum Transformer
This work addresses noise issues in quantum machine learning for researchers in quantum computing, offering an incremental improvement with a novel architecture.
The paper tackles the vulnerability of Quantum Transformers to QPU noise by proposing the Vectorized Quantum Transformer (VQT), which uses vectorized nonlinear quantum encoding for efficient training and achieves competitive results in NLP benchmarks on IBM and IonQ quantum processors.
Vectorized quantum block encoding provides a way to embed classical data into Hilbert space, offering a pathway for quantum models, such as Quantum Transformers (QT), that replace classical self-attention with quantum circuit simulations to operate more efficiently. Current QTs rely on deep parameterized quantum circuits (PQCs), rendering them vulnerable to QPU noise, and thus hindering their practical performance. In this paper, we propose the Vectorized Quantum Transformer (VQT), a model that supports ideal masked attention matrix computation through quantum approximation simulation and efficient training via vectorized nonlinear quantum encoder, yielding shot-efficient and gradient-free quantum circuit simulation (QCS) and reduced classical sampling overhead. In addition, we demonstrate an accuracy comparison for IBM and IonQ in quantum circuit simulation and competitive results in benchmarking natural language processing tasks on IBM state-of-the-art and high-fidelity Kingston QPU. Our noise intermediate-scale quantum friendly VQT approach unlocks a novel architecture for end-to-end machine learning in quantum computing.