How to detect novelty in textual data streams? A comparative study of existing methods
This work addresses a data scarcity problem for researchers in text mining and novelty detection, but it is incremental as it focuses on benchmarking and simulation rather than proposing new methods.
The authors tackled the lack of datasets for novelty detection in textual data streams by creating a simulation framework to generate controlled datasets and benchmarking existing methods, finding that performance varies across scenarios and parameters, with experiments on the New York Times Annotated Dataset.
Since datasets with annotation for novelty at the document and/or word level are not easily available, we present a simulation framework that allows us to create different textual datasets in which we control the way novelty occurs. We also present a benchmark of existing methods for novelty detection in textual data streams. We define a few tasks to solve and compare several state-of-the-art methods. The simulation framework allows us to evaluate their performances according to a set of limited scenarios and test their sensitivity to some parameters. Finally, we experiment with the same methods on different kinds of novelty in the New York Times Annotated Dataset.