Rumour Evaluation with Very Large Language Models
This work addresses the spread of rumours on social media for public safety and information integrity, but it is incremental as it extends existing tasks with new methods.
The paper tackled the problem of digital misinformation by applying prompting-based large language models (GPT-3.5-turbo and GPT-4) to the RumourEval task for veracity prediction and stance classification on Twitter data, achieving substantial improvements in veracity prediction with best results outperforming prior ones by a margin, while stance classification showed comparable performance to prior results without improvement over fine-tuning methods.
Conversational prompt-engineering-based large language models (LLMs) have enabled targeted control over the output creation, enhancing versatility, adaptability and adhoc retrieval. From another perspective, digital misinformation has reached alarming levels. The anonymity, availability and reach of social media offer fertile ground for rumours to propagate. This work proposes to leverage the advancement of prompting-dependent LLMs to combat misinformation by extending the research efforts of the RumourEval task on its Twitter dataset. To the end, we employ two prompting-based LLM variants (GPT-3.5-turbo and GPT-4) to extend the two RumourEval subtasks: (1) veracity prediction, and (2) stance classification. For veracity prediction, three classifications schemes are experimented per GPT variant. Each scheme is tested in zero-, one- and few-shot settings. Our best results outperform the precedent ones by a substantial margin. For stance classification, prompting-based-approaches show comparable performance to prior results, with no improvement over finetuning methods. Rumour stance subtask is also extended beyond the original setting to allow multiclass classification. All of the generated predictions for both subtasks are equipped with confidence scores determining their trustworthiness degree according to the LLM, and post-hoc justifications for explainability and interpretability purposes. Our primary aim is AI for social good.