Entropy-based Exploration Conduction for Multi-step Reasoning
This work addresses a domain-specific problem for users of LLMs in complex reasoning tasks, offering an incremental improvement over existing methods.
The paper tackles the problem of automatically deciding exploration depth in multi-step reasoning with LLMs, which affects task performance and often incurs high cost and lacks flexibility. The proposed method, Entro-duction, dynamically adjusts depth using entropy features, achieving improved reasoning accuracy and exploration effectiveness across four benchmark datasets.
Multi-step processes via large language models (LLMs) have proven effective for solving complex reasoning tasks. However, the depth of exploration of the reasoning procedure can significantly affect the task performance. Existing methods to automatically decide the depth often lead to high cost and a lack of flexibility. To address these issues, we propose Entropy-based Exploration Depth Conduction (Entro-duction), a novel method that dynamically adjusts the exploration depth during multi-step reasoning by monitoring LLM's output entropy and variance entropy. We employ these two features to capture the model's uncertainty of the current step and the fluctuation of uncertainty across consecutive reasoning steps. Based on the observed entropy changes, the LLM selects whether to deepen, expand, or stop exploration according to the probability, which facilitates the trade-off between the reasoning accuracy and exploration effectiveness. Experimental results across four benchmark datasets demonstrate the efficacy of Entro-duction.