Exploring Multitask Learning for Low-Resource AbstractiveSummarization
This addresses the challenge of low-resource abstractive summarization for NLP applications, though the approach is incremental as it builds on established multitask learning techniques.
The paper tackles the problem of improving abstractive summarization performance with limited training data by exploring multitask learning with four auxiliary tasks. The results show that certain task combinations consistently outperform single-task models, with paraphrase detection being particularly beneficial across different architectures and corpora.
This paper explores the effect of using multitask learning for abstractive summarization in the context of small training corpora. In particular, we incorporate four different tasks (extractive summarization, language modeling, concept detection, and paraphrase detection) both individually and in combination, with the goal of enhancing the target task of abstractive summarization via multitask learning. We show that for many task combinations, a model trained in a multitask setting outperforms a model trained only for abstractive summarization, with no additional summarization data introduced. Additionally, we do a comprehensive search and find that certain tasks (e.g. paraphrase detection) consistently benefit abstractive summarization, not only when combined with other tasks but also when using different architectures and training corpora.