Enhancing Granular Sentiment Classification with Chain-of-Thought Prompting in Large Language Models
This work addresses the need for more nuanced sentiment analysis in user feedback for app developers, though it is incremental as it applies an existing prompting technique to a specific domain.
The paper tackled the problem of improving granular sentiment classification in app store reviews by using Chain-of-Thought prompting with large language models, resulting in an accuracy increase from 84% to 93% compared to simple prompting.
We explore the use of Chain-of-Thought (CoT) prompting with large language models (LLMs) to improve the accuracy of granular sentiment categorization in app store reviews. Traditional numeric and polarity-based ratings often fail to capture the nuanced sentiment embedded in user feedback. We evaluated the effectiveness of CoT prompting versus simple prompting on 2000 Amazon app reviews by comparing each method's predictions to human judgements. CoT prompting improved classification accuracy from 84% to 93% highlighting the benefit of explicit reasoning in enhancing sentiment analysis performance.