Asking the Right Questions: Improving Reasoning with Generated Stepping Stones
This addresses the problem of LLMs struggling with hard tasks that require multi-step reasoning, offering an incremental improvement to existing reasoning pipelines.
The paper tackles the challenge of improving LLMs' ability to solve complex reasoning tasks by generating intermediate stepping stones, such as simplifications or subproblems, and shows that this approach substantially helps LLMs of various capabilities in solving target tasks.
Recent years have witnessed tremendous progress in enabling LLMs to solve complex reasoning tasks such as math and coding. As we start to apply LLMs to harder tasks that they may not be able to solve in one shot, it is worth paying attention to their ability to construct intermediate stepping stones that prepare them to better solve the tasks. Examples of stepping stones include simplifications, alternative framings, or subproblems. We study properties and benefits of stepping stones in the context of modern reasoning LLMs via ARQ (\textbf{A}king the \textbf{R}ight \textbf{Q}uestions), our simple framework which introduces a question generator to the default reasoning pipeline. We first show that good stepping stone questions exist and are transferrable, meaning that good questions can be generated, and they substantially help LLMs of various capabilities in solving the target tasks. We next frame stepping stone generation as a post-training task and show that we can fine-tune LLMs to generate more useful stepping stones by SFT and RL on synthetic data.