CIDER: Context sensitive sentiment analysis for short-form text
This work addresses the issue of missing contextual meaning in sentiment analysis for researchers analyzing topic-specific short texts, though it is incremental as it builds on existing unsupervised methods.
The paper tackles the problem of context-insensitive sentiment analysis in short texts by introducing CIDER, a method that infers term valence from the corpus context, and demonstrates it outperforms state-of-the-art unsupervised techniques on a tweet dataset about weather.
Researchers commonly perform sentiment analysis on large collections of short texts like tweets, Reddit posts or newspaper headlines that are all focused on a specific topic, theme or event. Usually, general-purpose sentiment analysis methods are used. These perform well on average but miss the variation in meaning that happens across different contexts, for example, the word "active" has a very different intention and valence in the phrase "active lifestyle" versus "active volcano". This work presents a new approach, CIDER (Context Informed Dictionary and sEmantic Reasoner), which performs context-sensitive linguistic analysis, where the valence of sentiment-laden terms is inferred from the whole corpus before being used to score the individual texts. In this paper, we detail the CIDER algorithm and demonstrate that it outperforms state-of-the-art generalist unsupervised sentiment analysis techniques on a large collection of tweets about the weather. CIDER is also applicable to alternative (non-sentiment) linguistic scales. A case study on gender in the UK is presented, with the identification of highly gendered and sentiment-laden days. We have made our implementation of CIDER available as a Python package: https://pypi.org/project/ciderpolarity/.