CEM: Machine-Human Chatting Handoff via Causal-Enhance Module
31212Authors
nan
Abstract
AbstractAiming to ensure chatbot quality by predicting chatbot failure and enabling human-agent collaboration, Machine-Human Chatting Handoff (MHCH) has attracted lots of attention from both industry and academia in recent years. However, most existing methods mainly focus on the dialogue context or assist with global satisfaction prediction based on multi-task learning, which ignore the grounded relationships among the causal variables, like the user state and labor cost. These variables are significantly associated with handoff decisions, resulting in prediction bias and cost increasement. Therefore, we propose Causal-Enhance Module (CEM) by establishing the causal graph of MHCH based on these two variables, which is a simple yet effective module and can be easy to plug into the existing MHCH methods. For the impact of users, we use the user state to correct the prediction bias according to the causal relationship of multi-task. For the labor cost, we train an auxiliary cost simulator to calculate unbiased labor cost through counterfactual learning so that a model becomes cost-aware. Extensive experiments conducted on four real-world benchmarks demonstrate the effectiveness of CEM in generally improving the performance of existing MHCH methods without any elaborated model crafting.
Keywords
Full Content
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3242 - 3253 December 7-11, 2022 ©2022 Association for Computational Linguistics CEM: Machine-Human Chatting Handoff via Causal-Enhance Module Shanshan Zhong1, Jinghui Qin1, Zhongzhan Huang1, Daifeng Li2∗ 1 School of Computer Science and Engineering, Sun Yat-sen University 2 School of Information Management, Sun Yat-sen University {zhongshsh5,qinjingh,huangzhzh23}@mail2.sysu.edu.cn, lidaifeng@mail.sysu.edu.cn Abstract Aiming to ensure chatbot quality by predicting chatbot failure and enabling human-agent col- laboration, Machine-Human Chatting Handoff (MHCH) has attracted lots of attention from both industry and academia in recent years. However, most existing methods mainly focus on the dialogue context or assist with global sat- isfaction prediction based on multi-task learn- ing, which ignore the grounded relationships among the causal variables, like the user state and labor cost. These variables are significantly associated with handoff decisions, resulting in prediction bias and cost increase. Therefore, we propose Causal-Enhance Module (CEM) by establishing the causal graph of MHCH based on these two variables, which is a simple yet effective module and can be easy to plug into the existing MHCH methods. For the impact of users, we use the user state to correct the prediction bias according to the causal relation- ship of multi-task. For the labor cost, we train an auxiliary cost simulator to calculate unbi- ased labor cost through counterfactual learning so that a model becomes cost-aware. Exten- sive experiments conducted on four real-world benchmarks demonstrate the effectiveness of CEM in generally improving the performance of existing MHCH methods without any elabo- rated model crafting. 1 Introduction In recent years, with the rapid development of deep learning (He et al., 2016; Ren et al., 2015), more and more service-oriented organizations have de- ployed chatbots to alleviate the problem of limited service resources. Although these chatbots can re- spond in real-time and save labor cost, they suffer from inappropriate responses and invalid conversa- tions due to the limited quantity of available high- quality training data and the inherent biases (Xu et al., 2019; Liang et al., 2022) of neural networks. ∗_Corresponding author: lidaifeng@mail.sysu.edu.cn I want to buy a ticket to Chicago. Hi,what can I do for you? Hi, what can I do for you? Chatbot User When are you going to leave? Today at 5pm. User User I have to go today! Chatbot Sorry, the tickets have been sold out. Sorry, the tickets have been sold out. User Help me please!! Customer Service Global satisfaction: ...... User I want to buy a ticket to Chicago Today. Local sentiment Handoff label Normal Transferable Chatbot Chatbot Figure 1: An example of MHCH. Handoff label includes two types "normal" & "transferable", which denotes whether the chatbot should be transferred to human service. Moreover, human utterances sometimes are elu- sive since they are rich in acronyms, slang words, and even content without logic or grammar, which are too obscure for a chatbot to comprehend. To alleviate these drawbacks, researchers introduced a human-agent collaboration mechanism named Machine-Human Chatting Handoff (MHCH) to al- low a human …
Content truncated to 500 words
Labeled Text
AbstractAiming to ensure chatbot quality by predicting chatbot failure and enabling human-agent collaboration, Machine-Human Chatting Handoff (MHCH) has attracted lots of attention from both industry and academia in recent years. However, most existing methods mainly focus on the dialogue context or assist with global satisfaction prediction based on multi-task learning, which ignore the grounded relationships among the causal variables, like the user state and labor cost. These variables are significantly associated with handoff decisions, resulting in prediction bias and cost increasement. Therefore, we propose Causal-Enhance Module (CEM) by establishing the causal graph of MHCH based on these two variables, which is a simple yet effective module and can be easy to plug into the existing MHCH methods. For the impact of users, we use the user state to correct the prediction bias according to the causal relationship of multi-task. For the labor cost, we train an auxiliary cost simulator to calculate unbiased labor cost through counterfactual learning so that a model becomes cost-aware. Extensive experiments conducted on four real-world benchmarks demonstrate the effectiveness of CEM in generally improving the performance of existing MHCH methods without any elaborated model crafting.
Publication Details
- Paper ID:
- 31212
- Year:
- 2022
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan