Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
This work addresses the problem of enabling more flexible and faster reasoning in LLMs for researchers and practitioners, but it is incremental as it synthesizes existing research rather than introducing new methods.
This survey tackles the limitation of conventional Chain-of-Thought reasoning in LLMs, which relies on explicit language, by providing a comprehensive overview of latent CoT reasoning that embeds reasoning in latent spaces, establishing a systematic taxonomy and analyzing recent advances.
Large Language Models (LLMs) have shown impressive performance on complex tasks through Chain-of-Thought (CoT) reasoning. However, conventional CoT relies on explicitly verbalized intermediate steps, which constrains its broader applicability, particularly in abstract reasoning tasks beyond language. To address this, there has been growing research interest in \textit{latent CoT reasoning}, where the reasoning process is embedded within latent spaces. By decoupling reasoning from explicit language generation, latent CoT offers the promise of richer cognitive representations and facilitates more flexible, faster inference. This paper aims to present a comprehensive overview of this emerging paradigm and establish a systematic taxonomy. We analyze recent advances in methods, categorizing them from token-wise horizontal approaches to layer-wise vertical strategies. We then provide in-depth discussions of these methods, highlighting their design principles, applications, and remaining challenges. We hope that our survey provides a structured foundation for advancing this promising direction in LLM reasoning. The relevant papers will be regularly updated at https://github.com/EIT-NLP/Awesome-Latent-CoT.