Psychological State in Text: A Limitation of Sentiment Analysis
This work highlights a limitation in sentiment analysis for real-world psychological assessment, which is incremental as it builds on existing methods to test a specific application.
The study investigated whether sentiment analysis models can predict psychological states beyond simple positive/negative sentiment, finding that while the model performed well on its training task, its predictions showed no correlation with human self-reported sentiment scores.
Starting with the idea that sentiment analysis models should be able to predict not only positive or negative but also other psychological states of a person, we implement a sentiment analysis model to investigate the relationship between the model and emotional state. We first examine psychological measurements of 64 participants and ask them to write a book report about a story. After that, we train our sentiment analysis model using crawled movie review data. We finally evaluate participants' writings, using the pretrained model as a concept of transfer learning. The result shows that sentiment analysis model performs good at predicting a score, but the score does not have any correlation with human's self-checked sentiment.