Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner
This work addresses the explainability of large language models in QA, offering a method to generate step-by-step explanations, though it appears incremental as it builds on prior entailment tree approaches.
The authors tackled the problem of generating structured explanations (entailment trees) for question answering systems by proposing the Iterative Retrieval-Generation Reasoner (IRGR), which outperformed existing benchmarks on the EntailmentBank dataset with around 300% gain in overall correctness.
Large language models have achieved high performance on various question answering (QA) benchmarks, but the explainability of their output remains elusive. Structured explanations, called entailment trees, were recently suggested as a way to explain and inspect a QA system's answer. In order to better generate such entailment trees, we propose an architecture called Iterative Retrieval-Generation Reasoner (IRGR). Our model is able to explain a given hypothesis by systematically generating a step-by-step explanation from textual premises. The IRGR model iteratively searches for suitable premises, constructing a single entailment step at a time. Contrary to previous approaches, our method combines generation steps and retrieval of premises, allowing the model to leverage intermediate conclusions, and mitigating the input size limit of baseline encoder-decoder models. We conduct experiments using the EntailmentBank dataset, where we outperform existing benchmarks on premise retrieval and entailment tree generation, with around 300% gain in overall correctness.