Faithful Question Answering with Monte-Carlo Planning
This addresses the problem of interpretability and faithfulness in AI reasoning for users needing transparent decision-making, though it is incremental as it builds on existing modular and planning approaches.
The paper tackles the challenge of making large language models reveal faithful intermediate reasoning steps in question answering by proposing FAME, which organizes reasoning as structured entailment trees and uses Monte-Carlo planning to search for high-quality steps, achieving state-of-the-art performance on a standard benchmark with a smaller model size.
Although large language models demonstrate remarkable question-answering performances, revealing the intermediate reasoning steps that the models faithfully follow remains challenging. In this paper, we propose FAME (FAithful question answering with MontE-carlo planning) to answer questions based on faithful reasoning steps. The reasoning steps are organized as a structured entailment tree, which shows how premises are used to produce intermediate conclusions that can prove the correctness of the answer. We formulate the task as a discrete decision-making problem and solve it through the interaction of a reasoning environment and a controller. The environment is modular and contains several basic task-oriented modules, while the controller proposes actions to assemble the modules. Since the search space could be large, we introduce a Monte-Carlo planning algorithm to do a look-ahead search and select actions that will eventually lead to high-quality steps. FAME achieves state-of-the-art performance on the standard benchmark. It can produce valid and faithful reasoning steps compared with large language models with a much smaller model size.