A Semi-Supervised Stable Variational Network for Promoting Replier-Consistency in Dialogue Generation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1920–1930, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1920 A Semi-Supervised Stable Variational Network for Promoting Replier-Consistency in Dialogue Generation Jinxin Chang1,2,∗, Ruifang He1,2,3,∗†, Longbiao Wang1,2,†, Xiangyu Zhao1,2, Ting Yang1,2, and Ruifang Wang1,2 1Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin, China. 2College of Intelligence and Computing, Tianjin University, Tianjin, China. 3State Key Laboratory of Cognitive Intelligence, iFLYTEK, P.R. China. {changjinxin,rfhe,longbiao wang}@tju.edu.cn {zhaoxiangyu009,16622898776,wrf276224255}@163.com Abstract Neural sequence-to-sequence models for dia- log systems suffer from the problem of favor- ing uninformative and non replier-specific re- sponses due to lacking of the global and rele- vant information guidance. The existing meth- ods model the generation process by leverag- ing the neural variational network with simple Gaussian. However, the sampled information from latent space usually becomes useless due to the KL divergence vanishing issue, and the highly abstractive global variables easily di- lute the personal features of replier, leading to a non replier-specific response. Therefore, a novel Semi-Supervised Stable Variational Network (SSVN) is proposed to address these issues. We use a unit hypersperical distribu- tion, namely the von Mises-Fisher (vMF), as the latent space of a semi-supervised model, which can obtain the stable KL performance by setting a fixed variance and hence enhance the global information representation. Mean- while, an unsupervised extractor is introduced to automatically distill the replier-tailored fea- ture which is then injected into a supervised generator to encourage the replier-consistency. Experimental results on two large conversa- tion datasets show that our model outperforms the competitive baseline models significantly, and can generate diverse and replier-specific responses. 1 Introduction Dialog systems, aiming at generating relevant and fluent responses in the replier-consistent way, have received increasing attention due to its numerous applications (Grosz, 2016; Chen et al., 2017a). Recently, Seq2Seq neural networks (Sutskever et al., 2014) have demonstrated excellent results on open-domain conversation (Shang et al., 2015; ∗∗ Equal contribution. †† Corresponding author. Sordoni et al., 2015; Vinyals and V. Le, 2015; Yao et al., 2015). However, due to lacking of the global and relevant information guidance, they in- herently tend to generate trivial and uninforma- tive responses (e.g., “I don’t know”), rather than meaningful and replier-specific ones (Serban et al., 2016; Li et al., 2016). The existing methods based on neural vari- ational methods with Gaussian (Kingma and Welling, 2014; Kingma et al., 2014), are proposed to use a latent variable as the global information in decoder to strengthen the generation (Serban et al., 2017; Zhao et al., 2017; Chen et al., 2018). How- ever, they face the problems of (1) latent space futility and (2) replier-consistency decay. (1) The model tends to select more gain from a lower Kullback-Leibler (KL) divergence during training, which encourages the approximate pos- terior close to Gaussian prior, rendering the latent space of the former unused. Thus, the latent vari- ables on this space become worthless global guid- ance …

Content truncated to 500 words