SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets
This addresses the problem of analyzing sentiment in multilingual social media data for researchers and practitioners, but it is incremental as it builds on existing tasks and datasets.
The paper tackled sentiment analysis of code-mixed tweets by releasing annotated Hinglish and Spanglish corpora and reporting results from a SemEval-2020 competition, with best F1 scores of 75.0% for Hinglish and 80.6% for Spanglish.
In this paper, we present the results of the SemEval-2020 Task 9 on Sentiment Analysis of Code-Mixed Tweets (SentiMix 2020). We also release and describe our Hinglish (Hindi-English) and Spanglish (Spanish-English) corpora annotated with word-level language identification and sentence-level sentiment labels. These corpora are comprised of 20K and 19K examples, respectively. The sentiment labels are - Positive, Negative, and Neutral. SentiMix attracted 89 submissions in total including 61 teams that participated in the Hinglish contest and 28 submitted systems to the Spanglish competition. The best performance achieved was 75.0% F1 score for Hinglish and 80.6% F1 for Spanglish. We observe that BERT-like models and ensemble methods are the most common and successful approaches among the participants.