FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation
This work is significant for researchers and practitioners working with explainable AI, as it provides a more reliable method for generating natural language explanations from black-box models, particularly in NLP tasks like Question Answering and Fact Verification.
This paper introduces FiD-Ex, a model designed to improve sequence-to-sequence models for extractive rationale generation. It addresses issues like explanation fabrication, long input handling, and data scarcity, resulting in significant improvements in explanation metrics and task accuracy on the ERASER benchmark.
Natural language (NL) explanations of model predictions are gaining popularity as a means to understand and verify decisions made by large black-box pre-trained models, for NLP tasks such as Question Answering (QA) and Fact Verification. Recently, pre-trained sequence to sequence (seq2seq) models have proven to be very effective in jointly making predictions, as well as generating NL explanations. However, these models have many shortcomings; they can fabricate explanations even for incorrect predictions, they are difficult to adapt to long input documents, and their training requires a large amount of labeled data. In this paper, we develop FiD-Ex, which addresses these shortcomings for seq2seq models by: 1) introducing sentence markers to eliminate explanation fabrication by encouraging extractive generation, 2) using the fusion-in-decoder architecture to handle long input contexts, and 3) intermediate fine-tuning on re-structured open domain QA datasets to improve few-shot performance. FiD-Ex significantly improves over prior work in terms of explanation metrics and task accuracy, on multiple tasks from the ERASER explainability benchmark, both in the fully supervised and in the few-shot settings.