Skeleton-to-Response: Dialogue Generation Guided by Retrieval Memory

Proceedings of NAACL-HLT 2019, pages 1219–1228 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1219 Skeleton-to-Response: Dialogue Generation Guided by Retrieval Memory Deng Cai1∗, Yan Wang2†, Wei Bi2, Zhaopeng Tu2, Xiaojiang Liu2, Wai Lam1 and Shuming Shi2 1The Chinese University of Hong Kong, 2Tencent AI Lab thisisjcykcd@gmail.com, wlam@se.cuhk.edu.hk {brandenwang,victoriabi,zptu,kieranliu,shumingshi}@tencent.com Abstract Traditional generative dialogue models gener- ate responses solely from input queries. Such information is insufficient for generating a specific response since a certain query could be answered in multiple ways. Recently, re- searchers have attempted to fill the information gap by exploiting information retrieval tech- niques. For a given query, similar dialogues are retrieved from the entire training data and considered as an additional knowledge source. While the use of retrieval may harvest exten- sive information, the generative models could be overwhelmed, leading to unsatisfactory per- formance. In this paper, we propose a new framework which exploits retrieval results via a skeleton-to-response paradigm. At first, a skeleton is extracted from the retrieved dia- logues. Then, both the generated skeleton and the original query are used for response gen- eration via a novel response generator. Ex- perimental results show that our approach sig- nificantly improves the informativeness of the generated responses. 1 Introduction This paper focuses on tackling the challenges to develop a chit-chat style dialogue system (also known as chatbot). Chit-chat style dialogue sys- tem aims at giving meaningful and coherent re- sponses given a dialogue query in the open do- main. Most modern chit-chat systems can be cat- egorized into two categories, namely, information retrieval-based (IR) models and generative mod- els. The IR-based models (Ji et al., 2014; Hu et al., 2014) directly copy an existing response from a training corpus when receiving a response request. Since the training corpus is usually collected from real-world conversations and possibly post-edited ∗Work done while DC was interning at Tencent AI Lab. †Corresponding author. by a human, the retrieved responses are informa- tive and grammatical. However, the performance of such systems drops when a given dialogue his- tory is substantially different from those in the training corpus. The generative models (Shang et al., 2015; Vinyals and Le, 2015; Li et al., 2016a), on the other hand, generate a new utterance from scratch. While those generative models have better gen- eralization capacity in rare dialogue contexts, the generated responses tend to be universal and non- informative (e.g., “I don’t know”, “I think so” etc.) (Li et al., 2016a). It is partly due to the diversity of possible responses to a single query (i.e., the one-to-many problem). The dialogue query alone cannot decide a meaningful and specific response. Thus a well-trained model tends to generate the most frequent (safe but boring) responses instead. To summarize, IR-based models may give infor- mative but inappropriate responses while genera- tive models often do the opposite. It is desirable to combine both merits. Song et al. (2016) used an extra encoder for the retrieved response. The re- sulted dense representation, together with the …

Content truncated to 500 words