CiteSum: Citation Text-guided Scientific Extreme Summarization and Domain Adaptation with Limited Supervision
This work addresses the scalability issue in scientific extreme summarization by reducing reliance on human annotation, benefiting researchers and practitioners in natural language processing.
The authors tackled the problem of automatically generating ultra-short summaries (TLDR) for scientific papers by proposing a method to extract them from citation texts, creating a dataset 30 times larger than previous benchmarks. They demonstrated that models pre-trained on this dataset achieve state-of-the-art results in scientific and news summarization tasks with limited supervision, such as outperforming fully-supervised methods on SciTLDR with only 128 examples and improving zero-shot performance on XSum by +7.2 ROUGE-1.
Scientific extreme summarization (TLDR) aims to form ultra-short summaries of scientific papers. Previous efforts on curating scientific TLDR datasets failed to scale up due to the heavy human annotation and domain expertise required. In this paper, we propose a simple yet effective approach to automatically extracting TLDR summaries for scientific papers from their citation texts. Based on the proposed approach, we create a new benchmark CiteSum without human annotation, which is around 30 times larger than the previous human-curated dataset SciTLDR. We conduct a comprehensive analysis of CiteSum, examining its data characteristics and establishing strong baselines. We further demonstrate the usefulness of CiteSum by adapting models pre-trained on CiteSum (named CITES) to new tasks and domains with limited supervision. For scientific extreme summarization, CITES outperforms most fully-supervised methods on SciTLDR without any fine-tuning and obtains state-of-the-art results with only 128 examples. For news extreme summarization, CITES achieves significant gains on XSum over its base model (not pre-trained on CiteSum), e.g., +7.2 ROUGE-1 zero-shot performance and state-of-the-art few-shot performance. For news headline generation, CITES performs the best among unsupervised and zero-shot methods on Gigaword. Our dataset and code can be found at https://github.com/morningmoni/CiteSum.