Are Training Samples Correlated? Learning to Generate Dialogue Responses with Multiple References

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3826–3835 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3826 Are Training Samples Correlated? Learning to Generate Dialogue Responses with Multiple References Lisong Qiu1,2, Juntao Li1,2, Wei Bi3, Dongyan Zhao1,2, Rui Yan1,2∗ 1Center for Data Science, Peking University, Beijing, China 2Institute of Computer Science and Technology, Peking University, Beijing, China 3Tencent AI Lab, Shenzhen, China {qiuls,lijuntao,zhaody,ruiyan}@pku.edu.cn victoriabi@tencent.com Abstract Due to its potential applications, open-domain dialogue generation has become popular and achieved remarkable progress in recent years, but sometimes suffers from generic responses. Previous models are generally trained based on 1-to-1 mapping from an input query to its response, which actually ignores the nature of 1-to-n mapping in dialogue that there may exist multiple valid responses corresponding to the same query. In this paper, we pro- pose to utilize the multiple references by con- sidering the correlation of different valid re- sponses and modeling the 1-to-n mapping with a novel two-step generation architecture. The first generation phase extracts the common features of different responses which, com- bined with distinctive features obtained in the second phase, can generate multiple diverse and appropriate responses. Experimental re- sults show that our proposed model can ef- fectively improve the quality of response and outperform existing neural dialogue models on both automatic and human evaluations. 1 Introduction In recent years, open-domain dialogue genera- tion has become a research hotspot in Natural Language Processing due to its broad applica- tion prospect, including chatbots, virtual personal assistants, etc. Though plenty of systems have been proposed to improve the quality of gen- erated responses from various aspects such as topic (Xing et al., 2017), persona modeling (Zhang et al., 2018b) and emotion controlling (Zhou et al., 2018b), most of these recent approaches are pri- marily built upon the sequence-to-sequence archi- tecture (Cho et al., 2014; Shang et al., 2015) which suffers from the “safe” response problem (Li et al., 2016a; Sato et al., 2017). This can be ascribed to modeling the response generation process as 1- to-1 mapping, which ignores the nature of 1-to- ∗Corresponding author: Rui Yan (ruiyan@pku.edu.cn) Figure 1: An illustration of the two-step generation architecture. Different from the conventional meth- ods (shown in green color) which model each response from scratch every time, our method first builds a com- mon feature of multiple responses and models each re- sponse based on it afterward. n mapping of dialogue that multiple possible re- sponses can correspond to the same query. To deal with the generic response problem, various methods have been proposed, including diversity-promoting objective function (Li et al., 2016a), enhanced beam search (Shao et al., 2016), latent dialogue mechanism (Zhou et al., 2017, 2018a), Variational Autoencoders (VAEs) based models (Zhao et al., 2017; Serban et al., 2017), etc. However, these methods still view multiple responses as independent ones and fail to model multiple responses jointly. Recently, Zhang et al. (2018a) introduce a maximum likelihood strat- egy that given an …

Content truncated to 500 words