Modeling the Severity of Complaints in Social Media
This work addresses the need to understand complaint intent for applications like apology strategies, but it is incremental as it applies existing transformer methods to a new annotated dataset.
The paper tackled the problem of classifying complaint severity levels in social media for the first time in computational linguistics, achieving 55.7 macro F1 for severity classification and 88.2 macro F1 for binary complaint detection in a multi-task setting.
The speech act of complaining is used by humans to communicate a negative mismatch between reality and expectations as a reaction to an unfavorable situation. Linguistic theory of pragmatics categorizes complaints into various severity levels based on the face-threat that the complainer is willing to undertake. This is particularly useful for understanding the intent of complainers and how humans develop suitable apology strategies. In this paper, we study the severity level of complaints for the first time in computational linguistics. To facilitate this, we enrich a publicly available data set of complaints with four severity categories and train different transformer-based networks combined with linguistic information achieving 55.7 macro F1. We also jointly model binary complaint classification and complaint severity in a multi-task setting achieving new state-of-the-art results on binary complaint detection reaching up to 88.2 macro F1. Finally, we present a qualitative analysis of the behavior of our models in predicting complaint severity levels.