SemEval-2019 Task 8: Fact Checking in Community Question Answering Forums
This work addresses the challenge of automated fact checking for online communities, but it is incremental as it builds on existing competition frameworks without introducing new methods.
The paper tackled the problem of fact checking in community question answering forums by introducing a competition with two subtasks: classifying question types and predicting answer truthfulness, where systems improved over baselines for question classification but struggled with answer verification, with 17 and 11 submissions respectively.
We present SemEval-2019 Task 8 on Fact Checking in Community Question Answering Forums, which features two subtasks. Subtask A is about deciding whether a question asks for factual information vs. an opinion/advice vs. just socializing. Subtask B asks to predict whether an answer to a factual question is true, false or not a proper answer. We received 17 official submissions for subtask A and 11 official submissions for Subtask B. For subtask A, all systems improved over the majority class baseline. For Subtask B, all systems were below a majority class baseline, but several systems were very close to it. The leaderboard and the data from the competition can be found at http://competitions.codalab.org/competitions/20022