Generating Relevant and Coherent Dialogue Responses using Self-Separated Conditional Variational AutoEncoders

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 5624–5637 August 1–6, 2021. ©2021 Association for Computational Linguistics 5624 Generating Relevant and Coherent Dialogue Responses using Self-separated Conditional Variational AutoEncoders Bin Sun1, Shaoxiong Feng1, Yiwei Li1, Jiamou Liu2, Kan Li1∗ 1School of Computer Science & Technology, Beijing Institute of Technology 2School of Computer Science, The University of Auckland {binsun,shaoxiongfeng,liyiwei,likan}@bit.edu.cn jiamou.liu@auckland.ac.nz Abstract Conditional Variational AutoEncoder (CVAE) effectively increases the diversity and informa- tiveness of responses in open-ended dialogue generation tasks through enriching the context vector with sampled latent variables. However, due to the inherent one-to-many and many-to- one phenomena in human dialogues, the sam- pled latent variables may not correctly reflect the contexts’ semantics, leading to irrelevant and incoherent generated responses. To re- solve this problem, we propose Self-separated Conditional Variational AutoEncoder (abbre- viated as SepaCVAE) that introduces group information to regularize the latent variables, which enhances CVAE by improving the re- sponses’ relevance and coherence while main- taining their diversity and informativeness. SepaCVAE actively divides the input data into groups, and then widens the absolute difference between data pairs from distinct groups, while narrowing the relative distance between data pairs in the same group. Em- pirical results from automatic evaluation and detailed analysis demonstrate that SepaC- VAE can significantly boost responses in well- established open-domain dialogue datasets. 1 Introduction When conversing with a human user, an open- domain dialogue system is expected to generate human-like responses – responses that not only are diverse and informative, but also contain relevant and cohesive information that correctly addresses the context dialogue. Through using sampled latent variables, Conditional Variational AutoEncoders (CVAE) are powerful tools to ensure diversity and informativeness of the generated responses (Bow- man et al., 2016; Serban et al., 2017; Shen et al., 2017; Zhao et al., 2017; Chen et al., 2018). Yet, it is challenging for a CVAE-based dialogue gen- eration model to keep the responses relevant and c1 z1 c2 z2 Latent Space low similarity Semantic Space high similarity c1 c2 z1 z2 Latent Space low y similarity Semantic Space high hi h similarity c3 z3 high similarity low similarity Figure 1: In this example, the latent variables (z1, z2, z3) sampled by a general CVAE model don’t inherit the semantic relationship of the contexts (c1, c2, c3). Although c1 and c2 have a high similarity, the similarity between z1 and z2 is low. c2 and c3 have a low similarity, but z2 and z3 have a high similarity. coherent. The challenge arises as human dialogues inherently exhibit the one-to-many and many-to- one phenomena (Csaky et al., 2019), meaning that the same context could lead to very different re- sponses, and different contexts could lead to the same response, respectively. As a result, the la- tent variables sampled by CVAE often fail to cap- ture the correct contextual semantics, as shown in Fig. 1, leaving open the possibility that similar con- texts producing drastically different latent …

Content truncated to 500 words