Exploring Explainable Selection to Control Abstractive Summarization
This work addresses the problem of opaque and uncontrollable summarization models for users needing interpretable AI outputs, offering a novel approach with demonstrated performance gains.
The authors tackled the lack of explainability and control in neural abstractive summarization models by developing a select-and-generate framework that reveals sentence-level attributes, allowing users to guide summary generation. Their model, ESCA, outperformed eight state-of-the-art models on CNN/DailyMail and NYT50 datasets, as measured by ROUGE metrics and human evaluations.
Like humans, document summarization models can interpret a document's contents in a number of ways. Unfortunately, the neural models of today are largely black boxes that provide little explanation of how or why they generated a summary in the way they did. Therefore, to begin prying open the black box and to inject a level of control into the substance of the final summary, we developed a novel select-and-generate framework that focuses on explainability. By revealing the latent centrality and interactions between sentences, along with scores for sentence novelty and relevance, users are given a window into the choices a model is making and an opportunity to guide those choices in a more desirable direction. A novel pair-wise matrix captures the sentence interactions, centrality, and attribute scores, and a mask with tunable attribute thresholds allows the user to control which sentences are likely to be included in the extraction. A sentence-deployed attention mechanism in the abstractor ensures the final summary emphasizes the desired content. Additionally, the encoder is adaptable, supporting both Transformer- and BERT-based configurations. In a series of experiments assessed with ROUGE metrics and two human evaluations, ESCA outperformed eight state-of-the-art models on the CNN/DailyMail and NYT50 benchmark datasets.