THOUGHTSCULPT: Reasoning with Intermediate Revision and Search
This addresses the challenge of enhancing reasoning capabilities in AI systems, particularly for tasks requiring iterative refinement, and is incremental as it builds on existing search and reasoning methods.
The paper tackles the problem of improving reasoning and search for tasks with decomposable outputs by introducing THOUGHTSCULPT, a method that uses Monte Carlo Tree Search with revision actions, resulting in performance gains such as up to +30% interestingness in Story Outline Improvement and up to +16% word success rate in Mini-Crosswords Solving.
We present THOUGHTSCULPT, a general reasoning and search method for tasks with outputs that can be decomposed into components. THOUGHTSCULPT explores a search tree of potential solutions using Monte Carlo Tree Search (MCTS), building solutions one action at a time and evaluating according to any domain-specific heuristic, which in practice is often simply an LLM evaluator. Critically, our action space includes revision actions: THOUGHTSCULPT may choose to revise part of its previous output rather than continuing to build the rest of its output. Empirically, THOUGHTSCULPT outperforms state-of-the-art reasoning methods across three challenging tasks: Story Outline Improvement (up to +30% interestingness), Mini-Crosswords Solving (up to +16% word success rate), and Constrained Generation (up to +10% concept coverage).