Modeling the Severity of Complaints in Social Media

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2264–2274 June 6–11, 2021. ©2021 Association for Computational Linguistics 2264 Modeling the Severity of Complaints in Social Media Mali Jin Nikolaos Aletras Department of Computer Science University of Sheffield, UK {mjin6, n.aletras}@sheffield.ac.uk Abstract The speech act of complaining is used by humans to communicate a negative mismatch between reality and expectations as a reaction to an unfavorable sit- uation. Linguistic theory of pragmatics categorizes complaints into various severity levels based on the face-threat that the complainer is willing to under- take. This is particularly useful for understanding the intent of complainers and how humans develop suitable apology strategies. In this paper, we study the severity level of complaints for the first time in computational linguistics. To facilitate this, we enrich a publicly available data set of complaints with four severity categories and train different transformer-based networks combined with linguis- tic information achieving 55.7 macro F1. We also jointly model binary complaint classification and complaint severity in a multi-task setting achieving new state-of-the-art results on binary complaint de- tection reaching up to 88.2 macro F1. Finally, we present a qualitative analysis of the behavior of our models in predicting complaint severity levels.1,2 1 Introduction Complaining is a speech act that usually conveys negative emotions triggered by a discrepancy be- tween reality and expectations towards an entity or event (Olshtain and Weinbach, 1985). Complaints play an important role in human communication for expressing dissatisfaction. Based on complainers’ personalities and specific situations, expression of complaints vary from person to person (Vásquez, 2011). In pragmatics, complaints have been classified into various levels of severity according to their 1Data is available here: https://archive.org/ details/complaint_severity_data 2Code is available here: https://github.com/ mali726/Complaint-Severity emotional intensity, the amount of face-threat that the complainer is willing to undertake and their purpose (Olshtain and Weinbach, 1985; Trosborg, 2011; Kakolaki and Shahrokhi, 2016). Complain- ing purposes might include the expression of dissat- isfaction, to find solutions (e.g. ask for reparations) or both. Furthermore, a complaint can be catego- rized as implicit (i.e. without mentioning who is responsible) or explicit (i.e. accusing someone for doing something). Recent work on modeling complaints in natural language processing (NLP) has focused on distin- guishing complaints from non-complaints in social media (Preotiuc-Pietro et al., 2019; Jin and Aletras, 2020), however there is no previous study into more fine-grained complaint categories. Table 1 shows examples of social media posts expressing com- plaints grouped into four severity classes according to Trosborg (2011): (a) no explicit reproach; (b) disapproval; (c) accusation; and (d) blame. Identifying and analyzing the severity of com- plaints is important for: (a) improving customer ser- vice by recognizing the level of dissatisfaction and understanding complainers’ needs (Van Noort and Willemsen, 2012); (b) linguists to study the speech act of complaints in different levels of granularity on large scale (Tatsuki, 2000); and (c) developing downstream NLP applications such as automatic complaint response generation (Xu et al., 2017) …

Content truncated to 500 words