The Shmoop Corpus: A Dataset of Stories with Loosely Aligned Summaries
This provides a new dataset and benchmarks for story comprehension in NLP, addressing long-range dependencies in text, but it is incremental as it builds on existing summarization and QA tasks.
The authors introduced the Shmoop Corpus, a dataset of 231 stories with 7,234 chapters paired with chronologically aligned multi-paragraph summaries, to tackle story comprehension challenges in NLP. They constructed benchmarks like Cloze QA and abstractive summarization, showing that the alignment leads to significant improvements in learning-based methods.
Understanding stories is a challenging reading comprehension problem for machines as it requires reading a large volume of text and following long-range dependencies. In this paper, we introduce the Shmoop Corpus: a dataset of 231 stories that are paired with detailed multi-paragraph summaries for each individual chapter (7,234 chapters), where the summary is chronologically aligned with respect to the story chapter. From the corpus, we construct a set of common NLP tasks, including Cloze-form question answering and a simplified form of abstractive summarization, as benchmarks for reading comprehension on stories. We then show that the chronological alignment provides a strong supervisory signal that learning-based methods can exploit leading to significant improvements on these tasks. We believe that the unique structure of this corpus provides an important foothold towards making machine story comprehension more approachable.