Quantum Large Language Models via Tensor Network Disentanglers
This addresses performance limitations in LLMs for AI applications, but appears incremental as it builds on existing quantum and tensor network techniques.
The paper tackles enhancing Large Language Models by integrating quantum computing and tensor networks, achieving improved accuracy over classical models with low memory overhead.
We propose a method to enhance the performance of Large Language Models (LLMs) by integrating quantum computing and quantum-inspired techniques. Specifically, our approach involves replacing the weight matrices in the Self-Attention and Multi-layer Perceptron layers with a combination of two variational quantum circuits and a quantum-inspired tensor network, such as a Matrix Product Operator (MPO). This substitution enables the reproduction of classical LLM functionality by decomposing weight matrices through the application of tensor network disentanglers and MPOs, leveraging well-established tensor network techniques. By incorporating more complex and deeper quantum circuits, along with increasing the bond dimensions of the MPOs, our method captures additional correlations within the quantum-enhanced LLM, leading to improved accuracy beyond classical models while maintaining low memory overhead.