Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration
This addresses reasoning inefficiencies in LLMs, offering a scalable, model-agnostic solution, though it appears incremental as it builds on existing embedding and optimization techniques.
The paper tackles the problem of limited diversity and inefficient search in Large Language Models (LLMs) for complex reasoning by proposing Soft Reasoning, an embedding-based search framework that optimizes the first token's embedding to guide generation, resulting in improved reasoning accuracy and coherence with minimal computation.
Large Language Models (LLMs) struggle with complex reasoning due to limited diversity and inefficient search. We propose Soft Reasoning, an embedding-based search framework that optimises the embedding of the first token to guide generation. It combines (1) embedding perturbation for controlled exploration and (2) Bayesian optimisation to refine embeddings via a verifier-guided objective, balancing exploration and exploitation. This approach improves reasoning accuracy and coherence while avoiding reliance on heuristic search. Experiments demonstrate superior correctness with minimal computation, making it a scalable, model-agnostic solution. The code is released at https://github.com/alickzhu/Soft-Reasoning.