Learning to Abstract for Memory-augmented Conversational Response Generation

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3816–3825 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3816 Learning to Abstract for Memory-augmented Conversational Response Generation Zhiliang Tian,1,3∗ Wei Bi,2 Xiaopeng Li,1,3 Nevin L. Zhang1,3 1Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong 2Tencent AI Lab, Shenzhen, China ztianac@cse.ust.hk3HKUST-Xiaoi Joint Lab, Hong Kong victoriabi@tencent.com {xlibo,lzhang}@cse.ust.hk Abstract Neural generative models for open-domain chit-chat conversations have become an ac- tive area of research in recent years. A criti- cal issue with most existing generative models is that the generated responses lack informa- tiveness and diversity. A few researchers at- tempt to leverage the results of retrieval mod- els to strengthen the generative models, but these models are limited by the quality of the retrieval results. In this work, we pro- pose a memory-augmented generative model, which learns to abstract from the training cor- pus and saves the useful information to the memory to assist the response generation. Our model clusters query-response samples, ex- tracts characteristics of each cluster, and learns to utilize these characteristics for response generation. Experimental results show that our model outperforms other competitive base- lines. 1 Introduction Automatic human-computer dialogue / conversa- tion is a core topic in natural language process- ing. There is a boom in research on open-domain chit-chat dialogue systems due to the availability of vast conversational data online. Most existing models of dialogue systems can be divided into retrieval-based models and generative models. Given a query, retrieval-based (Ji et al., 2014) models search for the most similar query stored in the training corpus and directly copy its cor- responding response as the result. These mod- els cannot create new replies customized for the given queries. Generative models (Shang et al., 2015) learn a query-response mapping to generate responses by maximizing P(r|q), where q is the input query and r is the response. The most popu- lar generative model is the Sequence-to-Sequence ∗Work done while Zhiliang Tian was collaborating with Tencent AI Lab. Query Response Memory Where did you go for holiday? Fine weather today in Chicago! I like the weather today. There are many places for tour. Which city to travel next time? I traveled to Tibet. Sure, sunny in Chicago. It is too hot for me. Yes, especially museums. Maybe New York. (weather, today) (sunny, hot) Learn/to/Abstract Any places for traveling this weekend? Travel to New York’s museum for this weekend. Training/corpus Input Output (place, travel) (Tibet, New York, museum) key1 value1 key2 value2 𝑘? @ 𝑘A @ 𝑘@ @ 𝑘B @ 𝑣? @ 𝑣A @ 𝑣B @ 𝑣@ @ 𝑘? A 𝑘A A 𝑘@ A 𝑘B A 𝑣? A 𝑣A A 𝑣B A 𝑣@ A . . . . . . . . . . . . Figure 1: An example of abstracting training corpus and memorizing their characteristics in the form of key vectors and value vectors. Red and blue indicate two clusters. The …

Content truncated to 500 words