Typing Errors in Factual Knowledge Graphs: Severity and Possible Ways Out
This study addresses the problem of high typing error rates in widely used factual knowledge graphs, which impacts the reliability of AI research and downstream tasks for the AI community.
This paper investigates the quality of factual knowledge graphs (KGs) like DBpedia and Wikidata, revealing a significant typing error rate of 27% for coarse-grained types and up to 73% for fine-grained types. The authors propose an active typing error detection algorithm that leverages both gold and noisy labels to address this issue.
Factual knowledge graphs (KGs) such as DBpedia and Wikidata have served as part of various downstream tasks and are also widely adopted by artificial intelligence research communities as benchmark datasets. However, we found these KGs to be surprisingly noisy. In this study, we question the quality of these KGs, where the typing error rate is estimated to be 27% for coarse-grained types on average, and even 73% for certain fine-grained types. In pursuit of solutions, we propose an active typing error detection algorithm that maximizes the utilization of both gold and noisy labels. We also comprehensively discuss and compare unsupervised, semi-supervised, and supervised paradigms to deal with typing errors in factual KGs. The outcomes of this study provide guidelines for researchers to use noisy factual KGs. To help practitioners deploy the techniques and conduct further research, we published our code and data.