Topical Segmentation of Spoken Narratives: A Test Case on Holocaust Survivor Testimonies
This work addresses the problem of segmenting unstructured spoken narratives for Holocaust research and computational linguistics, though it is incremental as it builds on existing segmentation methods.
The paper tackled topical segmentation of unstructured spoken narratives, specifically Holocaust survivor testimonies, by hypothesizing that segment boundaries correspond to low mutual information between sentences and exploring generative Bayesian and neural methods, resulting in considerable improvements over previous work.
The task of topical segmentation is well studied, but previous work has mostly addressed it in the context of structured, well-defined segments, such as segmentation into paragraphs, chapters, or segmenting text that originated from multiple sources. We tackle the task of segmenting running (spoken) narratives, which poses hitherto unaddressed challenges. As a test case, we address Holocaust survivor testimonies, given in English. Other than the importance of studying these testimonies for Holocaust research, we argue that they provide an interesting test case for topical segmentation, due to their unstructured surface level, relative abundance (tens of thousands of such testimonies were collected), and the relatively confined domain that they cover. We hypothesize that boundary points between segments correspond to low mutual information between the sentences proceeding and following the boundary. Based on this hypothesis, we explore a range of algorithmic approaches to the task, building on previous work on segmentation that uses generative Bayesian modeling and state-of-the-art neural machinery. Compared to manually annotated references, we find that the developed approaches show considerable improvements over previous work.