Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation
This work addresses the need for more accurate and logically consistent summaries in natural language processing, though it is incremental by building on multi-task learning with auxiliary tasks.
The paper tackled the problem of improving abstractive summarization by ensuring summaries contain salient information and are logically entailed by the input document, achieving statistically significant improvements over state-of-the-art on CNN/DailyMail, Gigaword, and DUC-2002 datasets.
An accurate abstractive summary of a document should contain all its salient information and should be logically entailed by the input document. We improve these important aspects of abstractive summarization via multi-task learning with the auxiliary tasks of question generation and entailment generation, where the former teaches the summarization model how to look for salient questioning-worthy details, and the latter teaches the model how to rewrite a summary which is a directed-logical subset of the input document. We also propose novel multi-task architectures with high-level (semantic) layer-specific sharing across multiple encoder and decoder layers of the three tasks, as well as soft-sharing mechanisms (and show performance ablations and analysis examples of each contribution). Overall, we achieve statistically significant improvements over the state-of-the-art on both the CNN/DailyMail and Gigaword datasets, as well as on the DUC-2002 transfer setup. We also present several quantitative and qualitative analysis studies of our model's learned saliency and entailment skills.