Quantum Attention by Overlap Interference: Predicting Sequences from Classical and Many-Body Quantum Data
This work introduces a quantum method for attention mechanisms, potentially enabling faster sequence prediction in quantum computing applications, though it appears incremental as it adapts classical self-attention to quantum systems.
The authors tackled the problem of implementing self-attention in quantum computing by proposing a variational quantum self-attention (QSA) that uses interference of state overlaps for nonlinearity and directly outputs a Renyi-1/2 cross-entropy loss, achieving a gate complexity scaling of O(T d^2) compared to O(T^2 d) classically, with simulations showing it learns sequence prediction on classical and quantum many-body data.
We propose a variational quantum implementation of self-attention (QSA), the core operation in transformers and large language models, which predicts future elements of a sequence by forming overlap-weighted combinations of past data. At variance with previous approaches, our QSA realizes the required nonlinearity through interference of state overlaps and returns a Renyi-1/2 cross-entropy loss directly as the expectation value of an observable, avoiding the need to decode amplitude-encoded predictions into classical logits. Furthermore, QSA naturally accommodates a constrained, trainable data-embedding that ties quantum state overlaps to data-level similarities. We find a gate complexity dominant scaling O(T d^2) for QSA, versus O(T^2 d) classically, suggesting an advantage in the practical regime where the sequence length T dominates the embedding size d. In simulations, we show that our QSA-based quantum transformer learns sequence prediction on classical data and on many-body transverse-field Ising quantum trajectories, establishing trainable attention as a practical primitive for quantum dynamical modeling.