Low-Resource Response Generation with Template Prior

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1886–1897, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1886 Low-Resource Response Generation with Template Prior Ze Yang†, Wei Wu♦, Jian Yang†, Can Xu♦, Zhoujun Li†∗ †State Key Lab of Software Development Environment, Beihang University, Beijing, China ♦Microsoft Corporation, Beijing, China {tobey, jiaya, lizj}@buaa.edu.cn {wuwei, caxu}@microsoft.com Abstract We study open domain response generation with limited message-response pairs. The problem exists in real-world applications but is less explored by the existing work. Since the paired data now is no longer enough to train a neural generation model, we consider leveraging the large scale of unpaired data that are much easier to obtain, and propose re- sponse generation with both paired and un- paired data. The generation model is defined by an encoder-decoder architecture with tem- plates as prior, where the templates are es- timated from the unpaired data as a neural hidden semi-markov model. By this means, response generation learned from the small paired data can be aided by the semantic and syntactic knowledge in the large unpaired data. To balance the effect of the prior and the in- put message to response generation, we pro- pose learning the whole generation model with an adversarial approach. Empirical studies on question response generation and sentiment response generation indicate that when only a few pairs are available, our model can signif- icantly outperform several state-of-the-art re- sponse generation models in terms of both au- tomatic and human evaluation. 1 Introduction Human-machine conversation is a long-standing goal of artificial intelligence. Early dialogue sys- tems are designed for task completion with con- versations restricted in specific domains (Young et al., 2013). Recently, thanks to the advances in deep learning techniques (Sutskever et al., 2014; Vaswani et al., 2017) and the availability of large amounts of human conversation on the internet, building an open domain dialogue system with data-driven approaches has become the new fash- ion in the research of conversational AI. Such dia- ∗ Corresponding Author logue systems can generate reasonable responses without any needs on rules, and have powered products in the industry such as Amazon Alexa (Ram et al., 2018) and Microsoft XiaoIce (Shum et al., 2018). State-of-the-art open domain response genera- tion models are based on the encoder-decoder ar- chitecture (Vinyals and Le, 2015; Shang et al., 2015). On the one hand, with proper extensions to the vanilla structure, existing models now are able to naturally handle conversation contexts (Serban et al., 2016; Xing et al., 2018), and synthesize re- sponses with various styles (Wang et al., 2017), emotions (Zhou et al., 2018), and personas (Li et al., 2016a). On the other hand, all the exist- ing success of open domain response generation builds upon an assumption that the large scale of paired data (Shao et al., 2016) or conversation ses- sions (Sordoni et al., 2015) are available. In this work, we challenge this assumption by …

Content truncated to 500 words