FaNS: a Facet-based Narrative Similarity Metric
This addresses the need for more granular narrative similarity in tasks like event understanding, though it is incremental as it builds on existing facet-based and LLM approaches.
The paper tackled the problem of accurately identifying semantically similar narratives by proposing FaNS, a facet-based similarity metric that leverages 5W1H facets extracted with LLMs, resulting in a 37% higher correlation than traditional metrics on a dataset from AllSides.
Similar Narrative Retrieval is a crucial task since narratives are essential for explaining and understanding events, and multiple related narratives often help to create a holistic view of the event of interest. To accurately identify semantically similar narratives, this paper proposes a novel narrative similarity metric called Facet-based Narrative Similarity (FaNS), based on the classic 5W1H facets (Who, What, When, Where, Why, and How), which are extracted by leveraging the state-of-the-art Large Language Models (LLMs). Unlike existing similarity metrics that only focus on overall lexical/semantic match, FaNS provides a more granular matching along six different facets independently and then combines them. To evaluate FaNS, we created a comprehensive dataset by collecting narratives from AllSides, a third-party news portal. Experimental results demonstrate that the FaNS metric exhibits a higher correlation (37\% higher) than traditional text similarity metrics that directly measure the lexical/semantic match between narratives, demonstrating its effectiveness in comparing the finer details between a pair of narratives.