A Role-Selected Sharing Network for Joint Machine-Human Chatting Handoff and Service Satisfaction Analysis

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9731–9741 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9731 A Role-Selected Sharing Network for Joint Machine-Human Chatting Handoff and Service Satisfaction Analysis Jiawei Liu1, Kaisong Song2,3, Yangyang Kang3, Guoxiu He4, Zhuoren Jiang5, Changlong Sun3,5, Wei Lu1∗, Xiaozhong Liu6∗ 1Wuhan University, Wuhan, China 2Northeastern University, Shenyang, China 3Alibaba Group, Hangzhou, China 5Zhejiang University, Hangzhou, China4East China Normal University, Shanghai, China 6Worcester Polytechnic Institute, Worcester, USA {laujames2017,weilu}@whu.edu.cn, {kaisong.sks,yangyang.kangyy}@alibaba-inc.com, gxhe@fem.ecnu.edu.cn, jiangzhuoren@zju.edu.cn, changlong.scl@taobao.com, xliu14@wpi.edu Abstract Chatbot is increasingly thriving in different do- mains, however, because of unexpected dis- course complexity and training data sparse- ness, its potential distrust hatches vital appre- hension. Recently, Machine-Human Chatting Handoff (MHCH), predicting chatbot failure and enabling human-algorithm collaboration to enhance chatbot quality, has attracted in- creasing attention from industry and academia. In this study, we propose a novel model, Role- Selected Sharing Network (RSSN), which in- tegrates both dialogue satisfaction estimation and handoff prediction in one multi-task learn- ing framework. Unlike prior efforts in dialog mining, by utilizing local user satisfaction as a bridge, global satisfaction detector and hand- off predictor can effectively exchange critical information. Specifically, we decouple the re- lation and interaction between the two tasks by the role information after the shared encoder. Extensive experiments on two public datasets demonstrate the effectiveness of our model. 1 Introduction Chatbot, as one of the recent palpable AI excite- ments, has been widely adopted to reduce the cost of customer service (Qiu et al., 2017; Ram et al., 2018; Zhou et al., 2020). However, due to the com- plexity of human conversation, auto-chatbot can hardly meet all users’ needs, while its potential failure perceives skepticism. AI-enabled customer service, for instance, may trigger unexpected busi- ness losses because of chatbot failures (Radziwill and Benton, 2017; Rajendran et al., 2019). More- over, for chatbot adoption in sensitive areas, such as healthcare (Chung and Park, 2019) and criminal justice (Wang et al., 2020a), any subtle statistical miscalculation may trigger serious health and legal ∗ Corresponding authors. What a business! It has been a week! utter3 We will ship goods about a week after placing the order. Please be patient. utter2 Sorry, my dear customer. We have pushed the warehouse to ship as soon as possible, and we will compensate the freight for you. utter5 Fine, hope I can receive it soon. utter7 We will try our best to improve your shopping experience. Thank you for your understanding and patience. utter6 utter1 Hello, I placed my order about one week ago. When can it be shipped? I’m sorry for the inconvenience. utter4 Handoff Satisfaction Rating: Chatbot Human Local (utterance) satisfaction Global (dialogue) satisfaction Figure 1: A snippet of a moderately satisfied customer service dialogue. There is a satisfaction rating at the end of the conversation. The utterance with an orange background color denotes a transferable utterance. consequences. To address this problem, recently, scholars proposed new dialog mining tasks to auto- assess dialogue satisfaction, a.k.a. Service Sat- isfaction Analysis …

Content truncated to 500 words