Improving Multi-turn Dialogue Modelling with Utterance ReWriter

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 22–31 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 22 Improving Multi-turn Dialogue Modelling with Utterance ReWriter Hui Su1∗, Xiaoyu Shen2∗, Rongzhi Zhang3, Fei Sun4, Pengwei Hu5 Cheng Niu1 and Jie Zhou1 1Pattern Recognition Center, Wechat AI, Tencent Inc, China 2MPI Informatics & Spoken Language Systems (LSV), Saarland Informatics Campus 3Institute of Software, University of Chinese Academy of Science 4Alibaba Group 5IBM Research, China aaronsu@tencent.com,xshen@mpi-inf.mpg.de Abstract Recent research has made impressive progress in single-turn dialogue modelling. In the multi-turn setting, however, current models are still far from satisfactory. One major chal- lenge is the frequently occurred coreference and information omission in our daily con- versation, making it hard for machines to un- derstand the real intention. In this paper, we propose rewriting the human utterance as a pre-process to help multi-turn dialgoue mod- elling. Each utterance is first rewritten to re- cover all coreferred and omitted information. The next processing steps are then performed based on the rewritten utterance. To properly train the utterance rewriter, we collect a new dataset with human annotations and introduce a Transformer-based utterance rewriting archi- tecture using the pointer network. We show the proposed architecture achieves remarkably good performance on the utterance rewriting task. The trained utterance rewriter can be eas- ily integrated into online chatbots and brings general improvement over different domains.1 1 Introduction Dialogue systems have made dramatic progress in recent years, especially in single-turn chit-chat and FAQ matching (Shang et al., 2015; Ghazvinine- jad et al., 2018; Molino et al., 2018; Chen et al., 2019). Nonethless, multi-turn dialogue modelling still remains extremely challenging (Vinyals and Le, 2015; Serban et al., 2016, 2017; Shen et al., 2018a,b). The challenge is multi-sided. One most important difficulty is the frequently oc- curred coreference and information omission in our daily conversations, especially in pro-drop languages like Chinese or Japanese. From our pre- liminary study of 2,000 Chinese multi-turn con- ∗Both authors contributed equally. 1The code is available on https://github.com/ chin-gyou/dialogue-utterance-rewriter. Context 1 Utterance 1 Human: 梅西有多高？ (Translation) Human: How tall is Messi? Utterance 2 ChatBot: 官方说法他的身高是5英尺7英寸。 ChatBot: Officially he is 5ft 7 inches. Utterance 3 Human: 他和C罗谁是最好的球员？ Human: Who is the best, he or C.Ronaldo? Utterance 3′ Human: 梅西和C罗谁是最好的球员？ Human: Who is the best, Messi or C.Ronaldo? Context 2 Utterance 1 Human: 你最喜欢什么电影？ Human: What movie do you like most? Utterance 2 ChatBot: 泰坦尼克。 ChatBot: Titanic. Utterance 3 Human: 为什么呢？ Human: Why? Utterance 3′ Human: 为什么最喜欢泰坦尼克？ Human: Why do you like Titanic most? Table 1: An example of multi-turn dialogue. Each ut- terance 3 is rewritten into Utterance 3′. Green means coreference and blue means omission. versations, different degrees of coreference and omission exist in more than 70% of the utterances. Capturing the hidden intention beneath them re- quires deeper understanding of the dialogue con- text, which is difficult for current neural network- based systems. Table 1 shows two typical exam- ples in multi-turn dialogues. “他”(he) from Con- …

Content truncated to 500 words