Complaint Identification in Social Media with Transformer Networks

Proceedings of the 28th International Conference on Computational Linguistics, pages 1765–1771 Barcelona, Spain (Online), December 8-13, 2020 1765 Complaint Identification in Social Media with Transformer Networks Mali Jin Nikolaos Aletras Department of Computer Science, University of Sheffield {mjin6, n.aletras}@sheffield.ac.uk Abstract Complaining is a speech act extensively used by humans to communicate a negative inconsis- tency between reality and expectations. Previous work on automatically identifying complaints in social media has focused on using feature-based and task-specific neural network models. Adapting state-of-the-art pre-trained neural language models and their combinations with other linguistic information from topics or sentiment for complaint prediction has yet to be explored. In this paper, we evaluate a battery of neural models underpinned by transformer networks which we subsequently combine with linguistic information. Experiments on a publicly available data set of complaints demonstrate that our models outperform previous state-of-the-art methods by a large margin achieving a macro F1 up to 87. 1 Introduction Complaining is a basic speech act, usually triggered by a discrepancy between reality and expectations towards an entity or event (Olshtain and Weinbach, 1985; Cohen and Olshtain, 1993; Kowalski, 1996). Social media has become a popular platform for expressing complaints online (Preotiuc-Pietro et al., 2019) where customers can directly address companies regarding issues with services and products. Complaint detection aims to identify a breach of expectations in a given text snippet. However, the use of implicit and ironic expressions and accompaniment of other speech acts such as suggestions, criticism, warnings and threats (Pawar et al., 2015) make it a challenging task. Identifying and classifying com- plaints automatically is important for: (a) improving customer service chatbots (Coussement and Van den Poel, 2008; Lailiyah et al., 2017; Yang et al., 2019a); (b) linguists to analyze complaint characteristics on large scale (V´asquez, 2011; Kakolaki and Shahrokhi, 2016); and (c) psychologists to understand the behavior of humans that express complaints (Sparks and Browning, 2010). Previous work has focused on binary classification between complaints and non-complaints in various domains (Preotiuc-Pietro et al., 2019; Jin et al., 2013; Coussement and Van den Poel, 2008). Further- more, some studies have performed more fine-grained complaint classification. For instance, complaints directed to public authorities have been categorized based on their topics (Forster and Entrup, 2017; Mer- son and Mary, 2017) or the responsible departments (Laksana and Purwarianti, 2014; Gunawan et al., 2018; Tjandra et al., 2015). Other categorizations are based on possible hazards and risks (Bhat and Cu- lotta, 2017) as well as escalation likelihood (Yang et al., 2019a). Most of these previous studies have used supervised machine learning models with features extracted from text (e.g. bag-of-words, topics, fea- tures extracted from psycho-linguistic dictionaries) or task-specific neural models trained from scratch. Adapting state-of-the-art pre-trained neural language models based on transformer networks (Vaswani et al., 2017) such as BERT (Devlin et al., 2018) and XLNet (Yang et al., 2019b) has yet to be explored. In this paper, we focus on the binary classification of Twitter posts into complaints or not (2019). We adapt and evaluate a …

Content truncated to 500 words