Supervised Sentiment Classification with CNNs for Diverse SE Datasets
This work addresses the challenge of sentiment analysis for software engineering researchers and practitioners, offering an incremental improvement over existing methods.
The authors tackled the problem of poor sentiment analysis performance on software engineering texts by proposing a hierarchical CNN-LSTM model trained on pre-trained word vectors, achieving state-of-the-art accuracy improvements across five datasets and demonstrating that labeling a small sample and retraining yields better accuracy than unsupervised classifiers.
Sentiment analysis, a popular technique for opinion mining, has been used by the software engineering research community for tasks such as assessing app reviews, developer emotions in issue trackers and developer opinions on APIs. Past research indicates that state-of-the-art sentiment analysis techniques have poor performance on SE data. This is because sentiment analysis tools are often designed to work on non-technical documents such as movie reviews. In this study, we attempt to solve the issues with existing sentiment analysis techniques for SE texts by proposing a hierarchical model based on convolutional neural networks (CNN) and long short-term memory (LSTM) trained on top of pre-trained word vectors. We assessed our model's performance and reliability by comparing it with a number of frequently used sentiment analysis tools on five gold standard datasets. Our results show that our model pushes the state of the art further on all datasets in terms of accuracy. We also show that it is possible to get better accuracy after labelling a small sample of the dataset and re-training our model rather than using an unsupervised classifier.