Explaining Interactions Between Text Spans
This addresses the problem of interpretability in NLU for researchers and practitioners, but it is incremental as it builds on existing explanation methods by focusing on non-adjacent interactions.
The paper tackles the lack of human-annotated data for explaining interactions between non-adjacent text spans in NLU tasks by introducing SpanEx, a dataset for NLI and fact-checking, and presents an unsupervised method to extract such interactions from models, showing that models often rely on different span connections than humans.
Reasoning over spans of tokens from different parts of the input is essential for natural language understanding (NLU) tasks such as fact-checking (FC), machine reading comprehension (MRC) or natural language inference (NLI). However, existing highlight-based explanations primarily focus on identifying individual important tokens or interactions only between adjacent tokens or tuples of tokens. Most notably, there is a lack of annotations capturing the human decision-making process w.r.t. the necessary interactions for informed decision-making in such tasks. To bridge this gap, we introduce SpanEx, a multi-annotator dataset of human span interaction explanations for two NLU tasks: NLI and FC. We then investigate the decision-making processes of multiple fine-tuned large language models in terms of the employed connections between spans in separate parts of the input and compare them to the human reasoning processes. Finally, we present a novel community detection based unsupervised method to extract such interaction explanations from a model's inner workings.