LLM Guided Inductive Inference for Solving Compositional Problems
This addresses limitations in LLM reasoning for open-world tasks, though it appears incremental as an extension of existing modular approaches.
The paper tackles the problem of LLMs struggling with deep reasoning tasks requiring external knowledge, introducing REBEL which uses recursive problem decomposition and external tools specified by natural language descriptions, achieving improved performance on compositional problems.
While large language models (LLMs) have demonstrated impressive performance in question-answering tasks, their performance is limited when the questions require knowledge that is not included in the model's training data and can only be acquired through direct observation or interaction with the real world. Existing methods decompose reasoning tasks through the use of modules invoked sequentially, limiting their ability to answer deep reasoning tasks. We introduce a method, Recursion based extensible LLM (REBEL), which handles open-world, deep reasoning tasks by employing automated reasoning techniques like dynamic planning and forward-chaining strategies. REBEL allows LLMs to reason via recursive problem decomposition and utilization of external tools. The tools that REBEL uses are specified only by natural language description. We further demonstrate REBEL capabilities on a set of problems that require a deeply nested use of external tools in a compositional and conversational setting.