Detecting Manipulated Contents Using Knowledge-Grounded Inference
This addresses the challenge of fake news detection for real-time applications, though it is incremental as it builds on existing methods like RAG and LLMs.
The paper tackles the problem of detecting zero-day manipulated content by proposing Manicod, a tool that uses retrieval-augmented generation with large language models to source real-time contextual information, achieving an overall F1 score of 0.856 on a new dataset of 4270 manipulated news pieces and outperforming existing methods by up to 1.9x in F1 score.
The detection of manipulated content, a prevalent form of fake news, has been widely studied in recent years. While existing solutions have been proven effective in fact-checking and analyzing fake news based on historical events, the reliance on either intrinsic knowledge obtained during training or manually curated context hinders them from tackling zero-day manipulated content, which can only be recognized with real-time contextual information. In this work, we propose Manicod, a tool designed for detecting zero-day manipulated content. Manicod first sources contextual information about the input claim from mainstream search engines, and subsequently vectorizes the context for the large language model (LLM) through retrieval-augmented generation (RAG). The LLM-based inference can produce a "truthful" or "manipulated" decision and offer a textual explanation for the decision. To validate the effectiveness of Manicod, we also propose a dataset comprising 4270 pieces of manipulated fake news derived from 2500 recent real-world news headlines. Manicod achieves an overall F1 score of 0.856 on this dataset and outperforms existing methods by up to 1.9x in F1 score on their benchmarks on fact-checking and claim verification.