Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Method
This addresses misinformation detection for news consumers, but is incremental as it builds on existing prompting methods.
The paper tackles news claim verification by introducing a Hierarchical Step-by-Step (HiSS) prompting method for LLMs, which outperforms state-of-the-art supervised approaches and few-shot baselines on two public misinformation datasets.
While large pre-trained language models (LLMs) have shown their impressive capabilities in various NLP tasks, they are still under-explored in the misinformation domain. In this paper, we examine LLMs with in-context learning (ICL) for news claim verification, and find that only with 4-shot demonstration examples, the performance of several prompting methods can be comparable with previous supervised models. To further boost performance, we introduce a Hierarchical Step-by-Step (HiSS) prompting method which directs LLMs to separate a claim into several subclaims and then verify each of them via multiple questions-answering steps progressively. Experiment results on two public misinformation datasets show that HiSS prompting outperforms state-of-the-art fully-supervised approach and strong few-shot ICL-enabled baselines.