Dual Latent Variable Model for Low-Resource Natural Language Generation in Dialogue Systems

Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018), pages 21–30 Brussels, Belgium, October 31 - November 1, 2018. c⃝2018 Association for Computational Linguistics 21 Dual Latent Variable Model for Low-Resource Natural Language Generation in Dialogue Systems Van-Khanh Tran1,2 and Le-Minh Nguyen1 1Japan Advanced Institute of Science and Technology, JAIST 1-1 Asahidai, Nomi, Ishikawa, 923-1292, Japan {tvkhanh, nguyenml}@jaist.ac.jp 2University of Information and Communication Technology, ICTU Thai Nguyen University, Vietnam tvkhanh@ictu.edu.vn Abstract Recent deep learning models have shown im- proving results to natural language generation (NLG) irrespective of providing sufficient an- notated data. However, a modest training data may harm such models’ performance. Thus, how to build a generator that can utilize as much of knowledge from a low-resource set- ting data is a crucial issue in NLG. This paper presents a variational neural-based generation model to tackle the NLG problem of having limited labeled dataset, in which we integrate a variational inference into an encoder-decoder generator and introduce a novel auxiliary auto- encoding with an effective training procedure. Experiments showed that the proposed meth- ods not only outperform the previous models when having sufficient training dataset but also show strong ability to work acceptably well when the training data is scarce. 1 Introduction Natural language generation (NLG) plays an crit- ical role in Spoken dialogue systems (SDSs) with the NLG task is mainly to convert a meaning representation produced by the dialogue manager, i.e., dialogue act (DA), into natural language re- sponses. SDSs are typically developed for various specific domains, i.e., flight reservations (Levin et al., 2000), buying a tv or a laptop (Wen et al., 2015b), searching for a hotel or a restaurant (Wen et al., 2015a), and so forth. Such systems often require well-defined ontology datasets that are ex- tremely time-consuming and expensive to collect. There is, thus, a need to build NLG systems that can work acceptably well when the training data is in short supply. There are two potential solutions for above- mentioned problems, which are domain adapta- tion training and model designing for low-resource training. First, domain adaptation training which aims at learning from sufficient source domain a model that can perform acceptably well on a dif- ferent target domain with a limited labeled target data. Domain adaptation generally involves two different types of datasets, one from a source do- main and the other from a target domain. Despite providing promising results for low-resource set- ting problems, the methods still need an adequate training data at the source domain site. Second, model designing for low-resource set- ting has not been well studied in the NLG litera- ture. The generation models have achieved very good performances irrespective of providing suf- ficient labeled datasets (Wen et al., 2015b,a; Tran et al., 2017; Tran and Nguyen, 2017). However, small training data easily result in worse genera- tion models in the supervised learning methods. Thus, this paper presents an explicit way to con- struct an effective low-resource setting generator. In summary, we make the following contribu- …

Content truncated to 500 words