Named Entity Inclusion in Abstractive Text Summarization
This work addresses a specific drawback in abstractive text summarization for NLP applications, but it is incremental as it builds on existing models like RoBERTa and BART.
The paper tackled the problem of named entity omission in abstractive text summarization by proposing a custom pretraining objective to enhance model attention on named entities, resulting in improved precision and recall metrics for named entity inclusion.
We address the named entity omission - the drawback of many current abstractive text summarizers. We suggest a custom pretraining objective to enhance the model's attention on the named entities in a text. At first, the named entity recognition model RoBERTa is trained to determine named entities in the text. After that, this model is used to mask named entities in the text and the BART model is trained to reconstruct them. Next, the BART model is fine-tuned on the summarization task. Our experiments showed that this pretraining approach improves named entity inclusion precision and recall metrics.