X-FACT: A New Benchmark Dataset for Multilingual Fact Checking
This provides a new benchmark for evaluating multilingual fact-checking models, addressing the need for diverse language data in automated verification.
The authors introduced X-FACT, the largest multilingual dataset for fact-checking with claims in 25 languages labeled by experts, and developed automated models that achieved an F-score of around 40%, indicating it is a challenging benchmark.
In this work, we introduce X-FACT: the largest publicly available multilingual dataset for factual verification of naturally existing real-world claims. The dataset contains short statements in 25 languages and is labeled for veracity by expert fact-checkers. The dataset includes a multilingual evaluation benchmark that measures both out-of-domain generalization, and zero-shot capabilities of the multilingual models. Using state-of-the-art multilingual transformer-based models, we develop several automated fact-checking models that, along with textual claims, make use of additional metadata and evidence from news stories retrieved using a search engine. Empirically, our best model attains an F-score of around 40%, suggesting that our dataset is a challenging benchmark for evaluation of multilingual fact-checking models.