Adversarial Domain Adaptation for Variational Neural Language Generation in Dialogue Systems

Proceedings of the 27th International Conference on Computational Linguistics, pages 1205–1217 Santa Fe, New Mexico, USA, August 20-26, 2018. 1205 Adversarial Domain Adaptation for Variational Neural Language Generation in Dialogue Systems Van-Khanh Tran1,2 and Le-Minh Nguyen1 1Japan Advanced Institute of Science and Technology, JAIST 1-1 Asahidai, Nomi, Ishikawa, 923-1292, Japan {tvkhanh, nguyenml}@jaist.ac.jp 2University of Information and Communication Technology, ICTU Thai Nguyen University, Vietnam tvkhanh@ictu.edu.vn Abstract Domain Adaptation arises when we aim at learning from source domain a model that can per- form acceptably well on a different target domain. It is especially crucial for Natural Language Generation (NLG) in Spoken Dialogue Systems when there are sufficient annotated data in the source domain, but there is a limited labeled data in the target domain. How to effectively utilize as much of existing abilities from source domains is a crucial issue in domain adaptation. In this paper, we propose an adversarial training procedure to train a Variational encoder-decoder based language generator via multiple adaptation steps. In this procedure, a model is first trained on a source domain data and then fine-tuned on a small set of target domain utterances under the guidance of two proposed critics. Experimental results show that the proposed method can effec- tively leverage the existing knowledge in the source domain to adapt to another related domain by using only a small amount of in-domain data. 1 Introduction Traditionally, Spoken Dialogue Systems are typically developed for various specific domains, including: finding a hotel, searching a restaurant (Wen et al., 2015a), or buying a tv, laptop (Wen et al., 2015b), flight reservations (Levin et al., 2000), etc. Such system are often requiring a well-defined ontology, which is essentially a data structured representation that the dialogue system can converse about. Statistical approaches to multi-domain in SDS system have shown promising results in how to reuse data in a domain-scalable framework efficiently (Young et al., 2013). Mrkˇsi´c et al. (2015) addressed the question of multi-domain in the SDS belief tracking by training a general model and adapting it to each domain. Recently, Recurrent Neural Networks (RNNs) based methods have shown improving results in tack- ling the domain adaptation issue (Chen et al., 2015; Shi et al., 2015; Wen et al., 2016a; Wen et al., 2016b). Such generators have also achieved promising results when providing such adequate annotated datasets (Wen et al., 2015b; Wen et al., 2015a; Tran et al., 2017; Tran and Nguyen, 2017a; Tran and Nguyen, 2017b). More recently, the development of the variational autoencoder (VAE) framework (Kingma and Welling, 2013; Rezende and Mohamed, 2015) has paved the way for learning large-scale, directed la- tent variable models. This has brought considerable benefits to significant progress in natural language processing (Bowman et al., 2015; Miao et al., 2016; Purushotham et al., 2017; Mnih and Gregor, 2014), dialogue system (Wen et al., 2017; Serban et al., 2017). This paper presents an adversarial training procedure to train a variational neural language genera- tor via multiple adaptation steps, which enables the generator to learn more …

Content truncated to 500 words