Semantic Wave Functions: Exploring Meaning in Large Language Models Through Quantum Formalism
This work addresses the challenge of interpreting and manipulating LLMs for researchers in AI and linguistics, though it appears incremental as it builds on existing quantum analogies without demonstrating practical applications.
This paper tackles the problem of understanding semantic relationships in Large Language Models by proposing an analogy to quantum mechanics, resulting in a new theoretical framework that uses complex-valued embeddings and quantum-derived formalisms to model semantic ambiguity and evolution.
Large Language Models (LLMs) encode semantic relationships in high-dimensional vector embeddings. This paper explores the analogy between LLM embedding spaces and quantum mechanics, positing that LLMs operate within a quantized semantic space where words and phrases behave as quantum states. To capture nuanced semantic interference effects, we extend the standard real-valued embedding space to the complex domain, drawing parallels to the double-slit experiment. We introduce a "semantic wave function" to formalize this quantum-derived representation and utilize potential landscapes, such as the double-well potential, to model semantic ambiguity. Furthermore, we propose a complex-valued similarity measure that incorporates both magnitude and phase information, enabling a more sensitive comparison of semantic representations. We develop a path integral formalism, based on a nonlinear Schrödinger equation with a gauge field and Mexican hat potential, to model the dynamic evolution of LLM behavior. This interdisciplinary approach offers a new theoretical framework for understanding and potentially manipulating LLMs, with the goal of advancing both artificial and natural language understanding.