Variational Hierarchical User-based Conversation Model

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1941–1950, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1941 Variational Hierarchical User-based Conversation Model JinYeong Bak School of Computing KAIST jy.bak@kaist.ac.kr Alice Oh School of Computing KAIST alice.oh@kaist.edu Abstract Generating appropriate conversation re- sponses requires careful modeling of the utterances and speakers together. Some recent approaches to response generation model both the utterances and the speakers, but these approaches tend to generate responses that are overly tailored to the speakers. To overcome this limitation, we propose a new model with a stochastic variable designed to capture the speaker information and deliver it to the conversational context. An important part of this model is the network of speakers in which each speaker is connected to one or more conversational partner, and this network is then used to model the speakers better. To test whether our model generates more appropriate conversation responses, we build a new conversation corpus containing approximately 27,000 speakers and 770,000 conversations. With this corpus, we run experiments of generating conversational responses and compare our model with other state-of-the-art models. By automatic evaluation metrics and human evaluation, we show that our model outperforms other models in generating appropriate responses. An additional advantage of our model is that it generates better responses for various new user scenarios, for example when one of the speakers is a known user in our corpus but the partner is a new user. For replicability, we make available all our code and data1. 1 Introduction In conversation response generation, modeling the speakers in addition to the utterances is important for generating appropriate responses. Knowing in- formation about a speaker, such as her linguistic style or personal information can help predict her response, and knowing more about both speakers 1https://github.com/NoSyu/VHUCM from their previous conversations can help pre- dict the contents of their conversation. Some re- cent work models the speakers in addition to the utterances (Li et al., 2016b; Olabiyi et al., 2018), but these models tend to overly emphasize the speaker such that they generate very similar re- sponses even when the previous utterances are dif- ferent. Another difficult and under-tackled prob- lem in conversation response generation is the cold start problem, when the training data do not con- tain one or both of the speakers. It then becomes very difficult to predict the appropriate responses. In this paper, we propose Variational Hierar- chical User-based Conversation Model (VHUCM) which has a stochastic variable conditioned on the speakers and affects the context. We generate the stochastic variable from a prior distribution whose parameters are given by feed-forward neural net- works. The inputs of the neural networks are two speakers that are represented as vector embed- dings. Then, we use an RNN to infer the conversa- tional context from the utterances and the stochas- tic variable. During training, we sample the vari- able from the variational distribution whose inputs are …

Content truncated to 500 words