Detecting Egregious Conversations between Customers and Virtual Agents

Proceedings of NAACL-HLT 2018, pages 1802–1811 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Detecting Egregious Conversations between Customers and Virtual Agents Tommy Sandbank, Michal Shmueli-Scheuer, Jonathan Herzig, David Konopnicki IBM Research Haifa 31905, Israel {tommy,shmueli,hjon,davidko}@il.ibm.com John Richards, David Piorkowski IBM Research Yorktown Heights, NY USA ajtr@us.ibm.com, david.piorkowski@ibm.com Abstract Virtual agents are becoming a promi- nent channel of interaction in cus- tomer service. Not all customer in- teractions are smooth, however, and some can become almost comically bad. In such instances, a human agent might need to step in and salvage the conversation. Detecting bad conversa- tions is important since disappointing customer service may threaten cus- tomer loyalty and impact revenue. In this paper, we outline an approach to detecting such egregious conversations, using behavioral cues from the user, patterns in agent responses, and user- agent interaction. Using logs of two commercial systems, we show that us- ing these features improves the detec- tion F1-score by around 20% over us- ing textual features alone. In addi- tion, we show that those features are common across two quite different do- mains and, arguably, universal. 1 Introduction Automated conversational agents (chatbots) are becoming widely used for various tasks such as personal assistants or as customer ser- vice agents. Recent studies project that 80% of businesses plan to use chatbots by 20201, and that chatbots will power 85% of customer service interactions by the year 20202. This increasing usage is mainly due to advances in artificial intelligence and natural language processing (Hirschberg and Manning, 2015) 1http://read.bi/2gU0szG 2http://gtnr.it/2z428RS along with increasingly capable chat devel- opment environments, leading to improve- ments in conversational richness and robust- ness. Still, chatbots may behave extremely badly, leading to conversations so off-the-mark that only a human agent could step in and salvage them. Consequences of these failures may in- clude loss of customer goodwill and associ- ated revenue, and even exposure to litigation if the failures can be shown to include fraudu- lent claims. Due to the increasing prevalence of chatbots, even a small fraction of such egre- gious3 conversations could be problematic for the companies deploying chatbots and the providers of chatbot services. In this paper we study detecting these egre- gious conversations that can arise in numer- ous ways. For example, incomplete or in- ternally inconsistent training data can lead to false classification of user intent. Bugs in dia- log descriptions can lead to dead ends. Fail- ure to maintain adequate context can cause chatbots to miss anaphoric references. In the extreme case, malicious actors may provide heavily biased (e.g., the Tay chatbot4) or even hacked misbehaviors. In this article, we focus on customer care systems. In such setting, a conversation usu- ally becomes egregious due to a combination of the aforementioned problems. The result- ing customer frustration may not surface in easily detectable ways such as the appearance of all caps, shouting to a speech recognizer, or the use of profanity or extreme punctuation. Consequently, the chatbot will continue as if …

Content truncated to 500 words