Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1711–1721, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems Tsung-Hsien Wen, Milica Gaˇsi´c, Nikola Mrkˇsi´c, Pei-Hao Su, David Vandyke and Steve Young Cambridge University Engineering Department, Trumpington Street, Cambridge, CB2 1PZ, UK {thw28,mg436,nm480,phs26,djv27,sjy}@cam.ac.uk Abstract Natural language generation (NLG) is a critical component of spoken dialogue and it has a significant impact both on usabil- ity and perceived quality. Most NLG sys- tems in common use employ rules and heuristics and tend to generate rigid and stylised responses without the natural vari- ation of human language. They are also not easily scaled to systems covering mul- tiple domains and languages. This pa- per presents a statistical language gener- ator based on a semantically controlled Long Short-term Memory (LSTM) struc- ture. The LSTM generator can learn from unaligned data by jointly optimising sen- tence planning and surface realisation us- ing a simple cross entropy training crite- rion, and language variation can be eas- ily achieved by sampling from output can- didates. With fewer heuristics, an objec- tive evaluation in two differing test do- mains showed the proposed method im- proved performance compared to previ- ous methods. Human judges scored the LSTM system higher on informativeness and naturalness and overall preferred it to the other systems. 1 Introduction The natural language generation (NLG) compo- nent provides much of the persona of a spoken dialogue system (SDS), and it has a significant impact on a user’s impression of the system. As noted in Stent et al. (2005), a good generator usu- ally depends on several factors: adequacy, flu- ency, readability, and variation. Previous ap- proaches attacked the NLG problem in different ways. The most common and widely adopted today is the rule-based (or template-based) ap- proach (Cheyer and Guzzoni, 2007; Mirkovic and Cavedon, 2011). Despite its robustness and ade- quacy, the frequent repetition of identical, rather stilted, output forms make talking to a rule-based generator rather tedious. Furthermore, the ap- proach does not easily scale to large open domain systems(Young et al., 2013; Gaˇsi´c et al., 2014; Henderson et al., 2014). Hence approaches to NLG are required that can be readily scaled whilst meeting the above requirements. The trainable generator approach exemplified by the HALOGEN (Langkilde and Knight, 1998) and SPaRKy system (Stent et al., 2004) provides a possible way forward. These systems include specific trainable modules within the generation framework to allow the model to adapt to different domains (Walker et al., 2007), or reproduce cer- tain style (Mairesse and Walker, 2011). However, these approaches still require a handcrafted gen- erator to define the decision space within which statistics can be used for optimisation. The result- ing utterances are therefore constrained by the pre- defined syntax and any domain-specific colloquial responses must be added manually. More recently, corpus-based methods (Oh and Rudnicky, 2000; Mairesse and Young, 2014; Wen et al., 2015) have received attention as access to data …

Content truncated to 500 words