Overview of the Shared Task on Fake News Detection in Urdu at FIRE 2020
This is an incremental effort addressing fake news detection for Urdu language communities, providing a new benchmark dataset.
The paper tackled fake news detection in Urdu by organizing a shared task as a binary classification problem, resulting in a best-performing BERT-based system achieving an F-score of 0.90.
This overview paper describes the first shared task on fake news detection in Urdu language. The task was posed as a binary classification task, in which the goal is to differentiate between real and fake news. We provided a dataset divided into 900 annotated news articles for training and 400 news articles for testing. The dataset contained news in five domains: (i) Health, (ii) Sports, (iii) Showbiz, (iv) Technology, and (v) Business. 42 teams from 6 different countries (India, China, Egypt, Germany, Pakistan, and the UK) registered for the task. 9 teams submitted their experimental results. The participants used various machine learning methods ranging from feature-based traditional machine learning to neural networks techniques. The best performing system achieved an F-score value of 0.90, showing that the BERT-based approach outperforms other machine learning techniques