Using Customer Service Dialogues for Satisfaction Analysis with Context-Assisted Multiple Instance Learning

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 198–207, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 198 Using Customer Service Dialogues for Satisfaction Analysis with Context-Assisted Multiple Instance Learning Kaisong Song1, Lidong Bing1, Wei Gao2, Jun Lin1, Lujun Zhao1 Jiancheng Wang3, Changlong Sun1, Xiaozhong Liu4, Qiong Zhang1 1Alibaba Group, Hangzhou, China 2Victoria University of Wellington, Wellington, New Zealand 3Soochow University, Suzhou, China 4Indiana University Bloomington, USA {kaisong.sks,l.bing,linjun.lj,lujun.zlj,qz.zhang}@alibaba-inc.com,wei.gao@vuw.ac.nz jiancheng.wang@qq.com, changlong.scl@taobao.com, liu237@indiana.edu Abstract Customers ask questions and customer ser- vice staffs answer their questions, which is the basic service model via multi-turn customer service (CS) dialogues on E-commerce plat- forms. Existing studies fail to provide compre- hensive service satisfaction analysis, namely satisfaction polarity classification (e.g., well satisfied, met and unsatisfied) and sentimental utterance identification (e.g., positive, neutral and negative). In this paper, we conduct a pilot study on the task of service satisfaction analy- sis (SSA) based on multi-turn CS dialogues. We propose an extensible Context-Assisted Multiple Instance Learning (CAMIL) model to predict the sentiments of all the customer ut- terances and then aggregate those sentiments into service satisfaction polarity. After that, we propose a novel Context Clue Matching Mechanism (CCMM) to enhance the represen- tations of all customer utterances with their matched context clues, i.e., sentiment and rea- soning clues. We construct two CS dialogue datasets from a top E-commerce platform. Ex- tensive experimental results are presented and contrasted against a few previous models to demonstrate the efficacy of our model. 1 1 Introduction In the past decades, E-commerce platforms, such as Amazon.com and Taobao.com2, have evolved into the most comprehensive and prosper- ous business ecosystems. They not only deeply involve other traditional businesses such as pay- ment and logistics, but also largely transform ev- ery aspect of retailing. Taking the customer ser- vice on Taobao as an example, third-party retailers are always online to answer any question at any stage of pre-sale, sale and after-sale, through an 1We have released the dataset at https://github. com/songkaisong/ssa. 2Taobao is a top E-commerce platform in China. Customer Is anybody there, please? u1 I’m here. Server u2 I applied for a return yesterday, and have already sent the product. u3 OK. u4 How do you return the freight to me, I paid 10 CNY. u5 It is not a quality problem, so you should pay the freight. u6 Don’t you pay the freight？ The sleeves are too fat, isn’t it a quality problem? u7 Dear, this is not a quality problem. u8 If it is not a quality problem, what else is? u9 You means no sleeves? u10 Is it broken? u11 Positive utterance Neutral utterance Negative utterance Satisfaction0Rating: Figure 1: Customer utterance and context clues (i.e., sentiment and reasoning clues) alignments in multi- turn dialogue utterances of an unsatisfied customer ser- vice. The utterances (ui) with positive/neutral/negative sentiments are denoted by red/orange/blue boxes. instant messenger within the platform. The top- ics of relevant customer …

Content truncated to 500 words