Uncovering Latent Arguments in Social Media Messaging by Employing LLMs-in-the-Loop Strategy
This addresses the challenge of automating argument extraction for social media analysis, reducing reliance on manual methods, but it is incremental as it builds on existing LLM capabilities.
The paper tackled the problem of discovering latent arguments in social media messaging by proposing an LLMs-in-the-Loop strategy, applying it to datasets like climate and COVID-19 vaccine campaigns with 14k and 9k Facebook ads, and achieving results such as improved stance prediction in climate debates.
The widespread use of social media has led to a surge in popularity for automated methods of analyzing public opinion. Supervised methods are adept at text categorization, yet the dynamic nature of social media discussions poses a continual challenge for these techniques due to the constant shifting of the focus. On the other hand, traditional unsupervised methods for extracting themes from public discourse, such as topic modeling, often reveal overarching patterns that might not capture specific nuances. Consequently, a significant portion of research into social media discourse still depends on labor-intensive manual coding techniques and a human-in-the-loop approach, which are both time-consuming and costly. In this work, we study the problem of discovering arguments associated with a specific theme. We propose a generic LLMs-in-the-Loop strategy that leverages the advanced capabilities of Large Language Models (LLMs) to extract latent arguments from social media messaging. To demonstrate our approach, we apply our framework to contentious topics. We use two publicly available datasets: (1) the climate campaigns dataset of 14k Facebook ads with 25 themes and (2) the COVID-19 vaccine campaigns dataset of 9k Facebook ads with 14 themes. Additionally, we design a downstream task as stance prediction by leveraging talking points in climate debates. Furthermore, we analyze demographic targeting and the adaptation of messaging based on real-world events.