Finding Trolls Under Bridges: Preliminary Work on a Motif Detector
This work addresses the need for culturally-aware natural language processing by automating motif detection, which was previously done manually, but it is incremental as it is a preliminary report with ongoing development.
The paper tackles the problem of automatically detecting motifs in narratives, which are culturally significant recurring elements, by developing a preliminary system that uses an off-the-shelf metaphor detector as a feature, achieving an F1 score of 0.35 on motifs and a macro-average F1 of 0.21 across four categories.
Motifs are distinctive recurring elements found in folklore that have significance as communicative devices in news, literature, press releases, and propaganda. Motifs concisely imply a large constellation of culturally-relevant information, and their broad usage suggests their cognitive importance as touchstones of cultural knowledge, making their detection a worthy step toward culturally-aware natural language processing tasks. Until now, folklorists and others interested in motifs have only extracted motifs from narratives manually. We present a preliminary report on the development of a system for automatically detecting motifs. We briefly describe an annotation effort to produce data for training motif detection, which is on-going. We describe our in-progress architecture in detail, which aims to capture, in part, how people determine whether or not a motif candidate is being used in a motific way. This description includes a test of an off-the-shelf metaphor detector as a feature for motif detection, which achieves a F1 of 0.35 on motifs and a macro-average F1 of 0.21 across four categories which we assign to motif candidates.