Crowdsourcing for Beyond Polarity Sentiment Analysis A Pure Emotion Lexicon
This work addresses the problem of resource scarcity for nuanced sentiment analysis in NLP, offering an incremental improvement by replacing expert-based methods with crowdsourcing.
The paper tackles the need for emotion lexicons in sentiment analysis beyond polarity by proposing a scalable, cost-effective crowdsourcing method for lexicon acquisition, eliminating the need for experts or gold standards, and compares crowd and expert evaluations to assess lexicon quality and crowd capabilities.
Sentiment analysis aims to uncover emotions conveyed through information. In its simplest form, it is performed on a polarity basis, where the goal is to classify information with positive or negative emotion. Recent research has explored more nuanced ways to capture emotions that go beyond polarity. For these methods to work, they require a critical resource: a lexicon that is appropriate for the task at hand, in terms of the range of emotions it captures diversity. In the past, sentiment analysis lexicons have been created by experts, such as linguists and behavioural scientists, with strict rules. Lexicon evaluation was also performed by experts or gold standards. In our paper, we propose a crowdsourcing method for lexicon acquisition, which is scalable, cost-effective, and doesn't require experts or gold standards. We also compare crowd and expert evaluations of the lexicon, to assess the overall lexicon quality, and the evaluation capabilities of the crowd.