Auto-Patching: Enhancing Multi-Hop Reasoning in Language Models
This addresses the challenge of complex reasoning in LLMs for AI applications, but it is incremental as it builds on existing frameworks like PatchScopes and shows modest gains compared to methods like Chain-of-Thought prompting.
The paper tackles the problem of multi-hop reasoning in large language models by introducing Auto-Patch, a method that dynamically patches hidden states during inference, resulting in an improvement in solve rate from 18.45% to 23.63% on the MuSiQue dataset.
Multi-hop questions still stump large language models (LLMs), which struggle to link information across multiple reasoning steps. We introduce Auto-Patch, a novel method that dynamically patches hidden states during inference to enhance multi-hop reasoning in LLMs. Building on the PatchScopes framework, Auto-Patch selectively modifies internal representations using a learned classifier. Evaluated on the MuSiQue dataset, Auto-Patch improves the solve rate from 18.45\% (baseline) to 23.63~$\pm$~0.7\% (3 runs), narrowing the gap to Chain-of-Thought prompting (27.44\%). Our results highlight the potential of dynamic hidden state interventions for advancing complex reasoning in LLMs.