TrollsWithOpinion: A Dataset for Predicting Domain-specific Opinion Manipulation in Troll Memes
This work addresses the challenge of identifying harmful opinion manipulation in online memes, which can demean or bully individuals, but it is incremental as it builds on an existing dataset with new annotations.
The authors tackled the problem of detecting opinion manipulation in troll memes by creating a new dataset of 8,881 multimodal memes annotated for troll classification and opinion manipulation across three domains, and found that existing state-of-the-art methods achieved only a 0.37 weighted-average F1-score, highlighting the need for specialized techniques.
Research into the classification of Image with Text (IWT) troll memes has recently become popular. Since the online community utilizes the refuge of memes to express themselves, there is an abundance of data in the form of memes. These memes have the potential to demean, harras, or bully targeted individuals. Moreover, the targeted individual could fall prey to opinion manipulation. To comprehend the use of memes in opinion manipulation, we define three specific domains (product, political or others) which we classify into troll or not-troll, with or without opinion manipulation. To enable this analysis, we enhanced an existing dataset by annotating the data with our defined classes, resulting in a dataset of 8,881 IWT or multimodal memes in the English language (TrollsWithOpinion dataset). We perform baseline experiments on the annotated dataset, and our result shows that existing state-of-the-art techniques could only reach a weighted-average F1-score of 0.37. This shows the need for a development of a specific technique to deal with multimodal troll memes.