Predicting Users’ Negative Feedbacks in Multi-Turn Human-Computer Dialogues

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 713–722, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Predicting Users’ Negative Feedbacks in Multi-Turn Human-Computer Dialogues ∗ Xin Wang1, Jianan Wang2, Yuanchao Liu1, Xiaolong Wang1, Zhuoran Wang3 and Baoxun Wang3 1Harbin Institute of Technology, Harbin, China 2Shanghai Jiao Tong University, Shanghai, China 3Tricorn (Beijing) Technology Co., Ltd, Beijing, China 1{xwang,lyc,wangxl}@insun.hit.edu.cn 2{wangjianan}@sjtu.edu.cn 3{wangzhuoran, wangbaoxun}@trio.ai Abstract User experience is essential for human- computer dialogue systems. However, it is impractical to ask users to provide explicit feedbacks when the agents’ responses dis- please them. Therefore, in this paper, we explore to predict users’ imminent dissat- isfactions caused by intelligent agents by analysing the existing utterances in the di- alogue sessions. To our knowledge, this is the first work focusing on this task. Sev- eral possible factors that trigger negative emotions are modelled. A relation se- quence model (RSM) is proposed to en- code the sequence of appropriateness of current response with respect to the ear- lier utterances. The experimental results show that the proposed structure is effec- tive in modelling emotional risk (possi- bility of negative feedback) than existing conversation modelling approaches. Be- sides, strategies of obtaining distance su- pervision data for pre-training are also dis- cussed in this work. Balanced sampling with respect to the last response in the dis- tance supervision data are shown to be re- liable for data augmentation. 1 Introduction As an ideal interaction mode, the human-computer conversation technology has gradually been con- ducted into the practical systems, among which the automatic agents adopting the task-oriented conversation and open-domain chatting abilities have developed rapidly and even come to commer- cial application stage (see Duer1 and XiaoIce2). ∗The work was done when the first author was an intern at Tricorn (Beijing) Technology Co., Ltd. 1http://duer.baidu.com/ 2http://www.msxiaoice.com/ �������� ��������������������������������� �������� ��������������������������������� ����� ��������������������� ������� ��������������������� ������������������ ������������������������������������������ ���� ������������� ����� ����� ��� ��� �� ��� ��� �� �� �� Figure 1: A example of multi-turn human- computer dialogue. The responses are readable and relevant to the corresponding queries, but the user in bad mood probably feel antipathy towards the computer’s cheer. Basically, the essence of such commercial dia- logue agents is keeping user active, and for this purpose, it is critical to improve users’ satisfaction in non-task oriented chatting services. The reasons of users’ displeasure lies in the sessions. As shown in Figure 1, the emotional conflict between the last agent’s response and the user’s mood leads to the dissatisfaction. There- fore, the negative feedback from user could be pre- dicted according to the context utterances. Indeed, such prediction is of great necessity for improv- ing users’ satisfaction, since it is possible to ad- dress problems within systems only if there exist methodologies to locate problems and summarize reasons behind. Ideally, if a dialogue agent antic- ipates the occurrence of the user’s negative feed- back, it is able to avoid such situations by tak- ing any possible actions, e.g., switching to an- 713 other …

Content truncated to 500 words