Ta'keed: The First Generative Fact-Checking System for Arabic Claims
It addresses the problem of limited explainable fact-checking for Arabic users, representing an incremental advance by applying existing methods to a new language and dataset.
The paper tackles the lack of explainable fact-checking systems for Arabic by introducing Ta'keed, which assesses claim truthfulness using retrieved snippets and generates explanations, achieving an F1 score of 0.72 in classification and an average cosine similarity of 0.76 for explanations.
This paper introduces Ta'keed, an explainable Arabic automatic fact-checking system. While existing research often focuses on classifying claims as "True" or "False," there is a limited exploration of generating explanations for claim credibility, particularly in Arabic. Ta'keed addresses this gap by assessing claim truthfulness based on retrieved snippets, utilizing two main components: information retrieval and LLM-based claim verification. We compiled the ArFactEx, a testing gold-labelled dataset with manually justified references, to evaluate the system. The initial model achieved a promising F1 score of 0.72 in the classification task. Meanwhile, the system's generated explanations are compared with gold-standard explanations syntactically and semantically. The study recommends evaluating using semantic similarities, resulting in an average cosine similarity score of 0.76. Additionally, we explored the impact of varying snippet quantities on claim classification accuracy, revealing a potential correlation, with the model using the top seven hits outperforming others with an F1 score of 0.77.