The State and Fate of Summarization Datasets: A Survey
This work identifies gaps like low-resource language datasets and over-reliance on news domains, aiming to streamline future summarization research for the NLP community.
The authors surveyed 133 summarization datasets across over 100 languages to address disjointed annotation efforts and lack of common terminology, creating an ontology and web interface to improve resource discovery and research coherence.
Automatic summarization has consistently attracted attention due to its versatility and wide application in various downstream tasks. Despite its popularity, we find that annotation efforts have largely been disjointed, and have lacked common terminology. Consequently, it is challenging to discover existing resources or identify coherent research directions. To address this, we survey a large body of work spanning 133 datasets in over 100 languages, creating a novel ontology covering sample properties, collection methods and distribution. With this ontology we make key observations, including the lack in accessible high-quality datasets for low-resource languages, and the field's over-reliance on the news domain and on automatically collected distant supervision. Finally, we make available a web interface that allows users to interact and explore our ontology and dataset collection, as well as a template for a summarization data card, which can be used to streamline future research into a more coherent body of work.