Narrative Incoherence Detection
This task is significant for researchers working on inter-sentential semantic understanding, as it provides a new benchmark for evaluating models' ability to detect semantic discrepancies in narrative flow. It is an initial step towards a new problem.
This paper introduces the task of narrative incoherence detection, focusing on identifying missing or discordant sentences within multi-sentence narratives. The authors implement token-level and sentence-level baselines, finding that token-level models perform better on shorter narratives, while sentence-level models excel on longer narratives and benefit from pre-training and auxiliary sentence prediction.
We propose the task of narrative incoherence detection as a new arena for inter-sentential semantic understanding: Given a multi-sentence narrative, decide whether there exist any semantic discrepancies in the narrative flow. Specifically, we focus on the missing sentence and discordant sentence detection. Despite its simple setup, this task is challenging as the model needs to understand and analyze a multi-sentence narrative, and predict incoherence at the sentence level. As an initial step towards this task, we implement several baselines either directly analyzing the raw text (\textit{token-level}) or analyzing learned sentence representations (\textit{sentence-level}). We observe that while token-level modeling has better performance when the input contains fewer sentences, sentence-level modeling performs better on longer narratives and possesses an advantage in efficiency and flexibility. Pre-training on large-scale data and auxiliary sentence prediction training objective further boost the detection performance of the sentence-level model.