Improving Agreement and Disagreement Identification in Online Discussions with A Socially-Tuned Sentiment Lexicon
This work addresses the problem of improving automated analysis of online discussions for researchers or platforms, but it is incremental as it builds on existing methods like CRFs and lexicons.
The paper tackled agreement and disagreement identification in online discussions by proposing an isotonic Conditional Random Fields model and a socially-tuned sentiment lexicon, achieving F1 scores of 0.74 and 0.67 for agreement and disagreement detection on Wikipedia Talk pages, outperforming state-of-the-art approaches.
We study the problem of agreement and disagreement detection in online discussions. An isotonic Conditional Random Fields (isotonic CRF) based sequential model is proposed to make predictions on sentence- or segment-level. We automatically construct a socially-tuned lexicon that is bootstrapped from existing general-purpose sentiment lexicons to further improve the performance. We evaluate our agreement and disagreement tagging model on two disparate online discussion corpora -- Wikipedia Talk pages and online debates. Our model is shown to outperform the state-of-the-art approaches in both datasets. For example, the isotonic CRF model achieves F1 scores of 0.74 and 0.67 for agreement and disagreement detection, when a linear chain CRF obtains 0.58 and 0.56 for the discussions on Wikipedia Talk pages.