Multi-Level Explanations for Generative Language Models
This addresses the problem of interpretability for users of large language models in tasks like summarization and question-answering, offering an incremental improvement over existing attribution methods.
The paper tackles the challenge of explaining generative language models in context-grounded tasks by proposing MExGen, a technique that scores context parts to quantify their influence on outputs, and shows it provides more faithful explanations than alternatives, including LLM self-explanations.
Despite the increasing use of large language models (LLMs) for context-grounded tasks like summarization and question-answering, understanding what makes an LLM produce a certain response is challenging. We propose Multi-Level Explanations for Generative Language Models (MExGen), a technique to provide explanations for context-grounded text generation. MExGen assigns scores to parts of the context to quantify their influence on the model's output. It extends attribution methods like LIME and SHAP to LLMs used in context-grounded tasks where (1) inference cost is high, (2) input text is long, and (3) the output is text. We conduct a systematic evaluation, both automated and human, of perturbation-based attribution methods for summarization and question answering. The results show that our framework can provide more faithful explanations of generated output than available alternatives, including LLM self-explanations. We open-source code for MExGen as part of the ICX360 toolkit: https://github$.$com/IBM/ICX360.