Automatically Identifying Complaints in Social Media

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5008–5019 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5008 Automatically Identifying Complaints in Social Media Daniel Preot¸iuc-Pietro Bloomberg LP dpreotiucpie@bloomberg.net Mihaela G˘aman Politehnica University of Bucharest mpgaman@gmail.com Nikolaos Aletras University of Sheffield n.aletras@sheffield.ac.uk Abstract Complaining is a basic speech act regularly used in human and computer mediated com- munication to express a negative mismatch be- tween reality and expectations in a particu- lar situation. Automatically identifying com- plaints in social media is of utmost impor- tance for organizations or brands to improve the customer experience or in developing dia- logue systems for handling and responding to complaints. In this paper, we introduce the first systematic analysis of complaints in computa- tional linguistics. We collect a new annotated data set of written complaints expressed in En- glish on Twitter.1 We present an extensive lin- guistic analysis of complaining as a speech act in social media and train strong feature-based and neural models of complaints across nine domains achieving a predictive performance of up to 79 F1 using distant supervision. 1 Introduction Complaining is a basic speech act used to express a negative mismatch between reality and expecta- tions towards a state of affairs, product, organiza- tion or event (Olshtain and Weinbach, 1987). Un- derstanding the expression of complaints in nat- ural language and automatically identifying them is of utmost importance for: (a) linguists to ob- tain a better understanding of the context, intent and types of complaints on a large scale; (b) psy- chologists to identify human traits underpinning complaint behavior and expression; (c) organiza- tions and advisers to improve the customer ser- vice by identifying and addressing client concerns and issues effectively in real time, especially on social media; (d) developing downstream natural language processing (NLP) applications, such as 1Data and code is available here: https: //github.com/danielpreotiuc/ complaints-social-media Tweet C S @FC Help hi, I ordered a necklace over a week ago and it still hasn’t arrived (...)  @BootsUK I love Boots! Shame you’re introduc- ing a man tax of 7% in 2018 :(   You suck  Table 1: Examples of tweets annotated for complaint (C) and sentiment (S). dialogue systems that aim to automatically iden- tify complaints. However, complaining has yet to be studied using computational approaches. The speech act of complaining, as previously defined in linguis- tics research (Olshtain and Weinbach, 1987) and adopted in this study, has as its core the concept of violated or breached expectations i.e., the person posting the complaint had their favorable expecta- tions breached by a party, usually the one to which the complaint is addressed. Complaints have been previously analyzed by linguists (V´asquez, 2011) as distinctly different from expressing negative sentiment towards an en- tity. Key to the definition of complaints is the ex- pression of the breach of expectations. Table 1 shows examples of tweets highlighting the differ- ences between complaints and sentiment. The first example expresses the …

Content truncated to 500 words