The Case for Claim Difficulty Assessment in Automatic Fact Checking
This addresses an overlooked issue in automated fact-checking research and practice, but it is incremental as it builds on existing work without introducing a new method.
The paper tackles the problem that some claims are more difficult to fact-check than others, identifying distinct types of difficulty through manual analysis and proposing a new task for claim difficulty prediction.
Fact-checking is the process of evaluating the veracity of claims (i.e., purported facts). In this opinion piece, we raise an issue that has received little attention in prior work -- that some claims are far more difficult to fact-check than others. We discuss the implications this has for both practical fact-checking and research on automated fact-checking, including task formulation and dataset design. We report a manual analysis undertaken to explore factors underlying varying claim difficulty and identify several distinct types of difficulty. We motivate this new claim difficulty prediction task as beneficial to both automated fact-checking and practical fact-checking organizations.