Current Landscape of the Russian Sentiment Corpora
This work provides a comparative analysis and ranking of Russian sentiment corpora, which is useful for researchers and practitioners in natural language processing, but it is incremental as it builds on existing methods and data.
The paper surveys over a dozen Russian-language sentiment analysis corpora, ranking them by annotation quality and investigating how training dataset choice affects model performance using BERT, finding that model quality generally improves with more training corpora.
Currently, there are more than a dozen Russian-language corpora for sentiment analysis, differing in the source of the texts, domain, size, number and ratio of sentiment classes, and annotation method. This work examines publicly available Russian-language corpora, presents their qualitative and quantitative characteristics, which make it possible to get an idea of the current landscape of the corpora for sentiment analysis. The ranking of corpora by annotation quality is proposed, which can be useful when choosing corpora for training and testing. The influence of the training dataset on the performance of sentiment analysis is investigated based on the use of the deep neural network model BERT. The experiments with review corpora allow us to conclude that on average the quality of models increases with an increase in the number of training corpora. For the first time, quality scores were obtained for the corpus of reviews of ROMIP seminars based on the BERT model. Also, the study proposes the task of the building a universal model for sentiment analysis.