Modeling Personalization in Continuous Space for Response Generation via Augmented Wasserstein Autoencoders

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1931–1940, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1931 Modeling Personalization in Continuous Space for Response Generation via Augmented Wasserstein Autoencoders Zhangming Chan1,3,∗, Juntao Li1,3,∗, Xiaopeng Yang4, Xiuying Chen1,3, Wenpeng Hu2,3, Dongyan Zhao1,3 and Rui Yan1,3,† 1 Center for Data Science, AAIS, Peking University 2 School of Mathematical Sciences, Peking University 3 Wangxuan Institute of Computer Technology, Peking University 4 David R. Cheriton School of Computer Science, Faculty of Mathematics, University of Waterloo {zhangming.chan,ljt,ruiyan}@pku.edu.cn Abstract Variational autoencoders (VAEs) and Wasser- stein autoencoders (WAEs) have achieved no- ticeable progress in open-domain response generation. Through introducing latent vari- ables in continuous space, these models are capable of capturing utterance-level seman- tics, e.g., topic, syntactic properties, and thus can generate informative and diversified re- sponses. In this work, we improve the WAE for response generation. In addition to the utterance-level information, we also model user-level information in latent con- tinue space. Specifically, we embed user-level and utterance-level information into two mul- timodal distributions, and combine these two multimodal distributions into a mixed distribu- tion. This mixed distribution will be used as the prior distribution of WAE in our proposed model, named as PersonaWAE. Experimental results on a large-scale real-world dataset con- firm the superiority of our model for gener- ating informative and personalized responses, where both automatic and human evaluations outperform state-of-the-art models. 1 Introduction Over the past decade, a myriad of conversational systems have been proposed in the field of artifi- cial intelligence and achieved remarkable success in various industry scenarios, such as e-commerce assistant (Li et al., 2017) and chit-chat machine XiaoIce (Shum et al., 2018). Based on the do- mains involved in previous research, existing work can be categorized into two groups, i.e., vertical- domain (Glas et al., 2015) and open-domain (Zhao et al., 2017), where the former group pursues to complete a specific target with limited domain knowledge while the latter one involves massive topics in conversations. In this work, we focus on ∗Equal contribution. Ordering is decided by a coin flip. † Corresponding author. the latter one and intend to generate a natural and meaningful response for a given conversation con- text. Most recent works build upon the sequence to sequence model (Bahdanau et al., 2014) and can generate a fluent response. But they suffer from the notorious “universal response” issue, i.e., gen- erating safe and uninformative responses (e.g., I don’t know) (Li et al., 2015). To address the aforementioned shortcoming, advanced conversational systems propose to cap- ture and incorporate extra information from two different levels, i.e., utterance-level and user-level. As for utterance-level information modeling, pre- vious works mainly construct models upon varia- tional autoencoders (VAE) (Kingma and Welling, 2014). By doing so, responses with diverse and informative words can be generated by introduc- ing a latent variable for modeling utterance-level information such as topic, and syntactic struc- ture (Bowman et …

Content truncated to 500 words