Overview of CLEF 2019 Lab ProtestNews: Extracting Protests from News in a Cross-context Setting
This work addresses the challenge of generalizable natural language processing for protest extraction from news, but it is incremental as it builds on existing lab frameworks and tasks.
The paper describes the CLEF-2019 Lab ProtestNews, which tackled the problem of extracting protest-related information from English news articles in a cross-country setting, involving tasks at document, sentence, and token levels. It found that neural networks performed best, but performance dropped significantly for most submissions when tested on data from China compared to India.
We present an overview of the CLEF-2019 Lab ProtestNews on Extracting Protests from News in the context of generalizable natural language processing. The lab consists of document, sentence, and token level information classification and extraction tasks that were referred as task 1, task 2, and task 3 respectively in the scope of this lab. The tasks required the participants to identify protest relevant information from English local news at one or more aforementioned levels in a cross-context setting, which is cross-country in the scope of this lab. The training and development data were collected from India and test data was collected from India and China. The lab attracted 58 teams to participate in the lab. 12 and 9 of these teams submitted results and working notes respectively. We have observed neural networks yield the best results and the performance drops significantly for majority of the submissions in the cross-country setting, which is China.