Are LLMs Good For Quantum Software, Architecture, and System Design?
For researchers and engineers in quantum computing, this work provides an initial assessment of LLMs' capabilities in this domain, revealing current limitations and guiding future efforts.
This paper evaluates nine frontier large language models (LLMs) on quantum computing problems, comparing their performance to graduate students. The results show that LLMs currently underperform graduate students, highlighting the need for further research and development to leverage LLMs in quantum software, architecture, and system design.
Quantum computers promise massive computational speedup for problems in many critical domains, such as physics, chemistry, cryptanalysis, healthcare, etc. However, despite decades of research, they remain far from entering an era of utility. The lack of mature software, architecture, and systems solutions capable of translating quantum-mechanical properties of algorithms into physical state transformations on qubit devices remains a key factor underlying the slow pace of technological progress. The problem worsens due to significant reliance on domain-specific expertise, especially for software developers, computer architects, and systems engineers. To address these limitations and accelerate large-scale high-performance quantum system design, we ask: Can large language models (LLMs) help with solving quantum software, architecture, and systems problems? In this work, we present a case study assessing the performance of LLMs on quantum system reasoning tasks. We evaluate nine frontier LLMs and compare their performance to graduate UT Austin students on a set of quantum computing problems. Finally, we recommend several directions along which research and engineering development efforts must be pursued.