Natural Language Generation for Spoken Dialogue System using RNN Encoder-Decoder Networks

Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 442–451, Vancouver, Canada, August 3 - August 4, 2017. c⃝2017 Association for Computational Linguistics Natural Language Generation for Spoken Dialogue System using RNN Encoder-Decoder Networks Van-Khanh Tran1,2 and Le-Minh Nguyen1 1Japan Advanced Institute of Science and Technology, JAIST 1-1 Asahidai, Nomi, Ishikawa, 923-1292, Japan {tvkhanh, nguyenml}@jaist.ac.jp 2University of Information and Communication Technology, ICTU Thai Nguyen University, Vietnam tvkhanh@ictu.edu.vn Abstract Natural language generation (NLG) is a critical component in a spoken dialogue system. This paper presents a Recurrent Neural Network based Encoder-Decoder architecture, in which an LSTM-based de- coder is introduced to select, aggregate se- mantic elements produced by an attention mechanism over the input elements, and to produce the required utterances. The proposed generator can be jointly trained both sentence planning and surface real- ization to produce natural language sen- tences. The proposed model was exten- sively evaluated on four different NLG datasets. The experimental results showed that the proposed generators not only con- sistently outperform the previous methods across all the NLG domains but also show an ability to generalize from a new, un- seen domain and learn from multi-domain datasets. 1 Introduction Natural Language Generation (NLG) plays a crit- ical role in Spoken Dialogue Systems (SDS) with task is to convert a meaning representation pro- duced by the Dialogue Manager into natural lan- guage utterances. Conventional approaches still rely on comprehensive hand-tuning templates and rules requiring expert knowledge of linguistic rep- resentation, including rule-based (Mirkovic et al., 2011), corpus-based n-gram models (Oh and Rud- nicky, 2000), and a trainable generator (Stent et al., 2004). Recently, Recurrent Neural Networks (RNNs) based approaches have shown promising perfor- mance in tackling the NLG problems. The RNN- based models have been applied for NLG as a joint training model (Wen et al., 2015a,b) and an end- to-end training model (Wen et al., 2016c). A re- curring problem in such systems is requiring anno- tated datasets for particular dialogue acts1 (DAs). To ensure that the generated utterance representing the intended meaning of the given DA, the previ- ous RNN-based models were further conditioned on a 1-hot vector representation of the DA. Wen et al. (2015a) introduced a heuristic gate to en- sure that all the slot-value pair was accurately cap- tured during generation. Wen et al. (2015b) sub- sequently proposed a Semantically Conditioned Long Short-term Memory generator (SC-LSTM) which jointly learned the DA gating signal and language model. More recently, Encoder-Decoder networks (Vinyals and Le, 2015; Li et al., 2015), especially the attentional based models (Wen et al., 2016b; Mei et al., 2015) have been explored to solve the NLG tasks. The Attentional RNN Encoder- Decoder (Bahdanau et al., 2014) (ARED) based approaches have also shown improved perfor- mance on a variety of tasks, e.g., image captioning (Xu et al., 2015; Yang et al., 2016), text summa- rization (Rush et al., 2015; Nallapati et al., 2016). While the RNN-based generators with DA gating-vector can prevent the undesirable seman- tic repetitions, …

Content truncated to 500 words