Enhancing Health Fact-Checking with LLM-Generated Synthetic Data
This work addresses the challenge of data scarcity for health fact-checking, which is crucial for combating misinformation, but it is incremental as it builds on existing methods with synthetic data augmentation.
The study tackled the problem of limited annotated training data for health-related fact-checking by proposing a synthetic data generation pipeline using large language models (LLMs) to augment training data, resulting in improved F1 scores by up to 0.019 on PubHealth and 0.049 on SciFact datasets.
Fact-checking for health-related content is challenging due to the limited availability of annotated training data. In this study, we propose a synthetic data generation pipeline that leverages large language models (LLMs) to augment training data for health-related fact checking. In this pipeline, we summarize source documents, decompose the summaries into atomic facts, and use an LLM to construct sentence-fact entailment tables. From the entailment relations in the table, we further generate synthetic text-claim pairs with binary veracity labels. These synthetic data are then combined with the original data to fine-tune a BERT-based fact-checking model. Evaluation on two public datasets, PubHealth and SciFact, shows that our pipeline improved F1 scores by up to 0.019 and 0.049, respectively, compared to models trained only on the original data. These results highlight the effectiveness of LLM-driven synthetic data augmentation in enhancing the performance of health-related fact-checkers.