Classifying Tweet Sentiment Using the Hidden State and Attention Matrix of a Fine-tuned BERTweet Model
This addresses sentiment analysis for social media data, but it's incremental as it builds on existing BERTweet methods.
The paper tackled tweet sentiment classification by engineering features from BERTweet's hidden states and attention matrices, achieving a validation accuracy of 0.9111.
This paper introduces a study on tweet sentiment classification. Our task is to classify a tweet as either positive or negative. We approach the problem in two steps, namely embedding and classifying. Our baseline methods include several combinations of traditional embedding methods and classification algorithms. Furthermore, we explore the current state-of-the-art tweet analysis model, BERTweet, and propose a novel approach in which features are engineered from the hidden states and attention matrices of the model, inspired by empirical study of the tweets. Using a multi-layer perceptron trained with a high dropout rate for classification, our proposed approach achieves a validation accuracy of 0.9111.