Chapter Captor: Text Segmentation in Novels
This addresses text segmentation for literary analysis and NLP applications, but is incremental as it builds on existing segmentation methods.
The researchers tackled the problem of segmenting long texts by predicting chapter boundaries in novels, achieving an F1-score of 0.453 for exact break prediction after creating a dataset of 9,126 English novels with an F1-score of 0.77 for chapter header recognition.
Books are typically segmented into chapters and sections, representing coherent subnarratives and topics. We investigate the task of predicting chapter boundaries, as a proxy for the general task of segmenting long texts. We build a Project Gutenberg chapter segmentation data set of 9,126 English novels, using a hybrid approach combining neural inference and rule matching to recognize chapter title headers in books, achieving an F1-score of 0.77 on this task. Using this annotated data as ground truth after removing structural cues, we present cut-based and neural methods for chapter segmentation, achieving an F1-score of 0.453 on the challenging task of exact break prediction over book-length documents. Finally, we reveal interesting historical trends in the chapter structure of novels.