Multi-task Learning for Natural Language Generation in Task-Oriented Dialogue

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1261–1266, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1261 Multi-task Learning for Natural Language Generation in Task-Oriented Dialogue Chenguang Zhu Michael Zeng Microsoft Speech and Dialogue Research Group, Redmond, WA, USA {chezhu, nzeng, xdh}@microsoft.com Xuedong Huang Abstract In task-oriented dialogues, Natural Language Generation (NLG) is the final and crucial step to produce user-facing system utterances. The result of NLG is directly related to the per- ceived quality and usability of a dialogue sys- tem. While most existing systems provide semantically correct responses given goals to present, they struggle to match the variation and fluency in the human language. In this paper, we propose a novel multi-task learning framework, NLG-LM, for natural language generation. In addition to generating high- quality responses conveying the required in- formation, it also explicitly targets for natu- ralness in generated responses via an uncon- ditioned language model. This can signifi- cantly improve the learning of style and vari- ation in human language. Empirical results show that this multi-task learning framework outperforms previous models across multiple datasets. For example, it improves the previ- ous best BLEU score on the E2E-NLG dataset by 2.2%, and on the Laptop dataset by 6.1%. 1 Introduction Natural Language Generation (NLG) is the fi- nal procedure in the pipeline of task-oriented di- alogues. As the result of NLG is directly fac- ing users, its readability and informativeness have a direct impact on users’ perception of the en- tire dialogue system. On one hand, the response must contain the desired information, referred to as meaning representation (MR), in order to pro- vide or request a user’s information. On the other hand, the system response needs to mimic the flu- ency and variation in human language to improve the user experience. To this end, there have been numerous studies on methods to generate natural responses for task-driven dialogues. Early work primarily employ predefined rules or syntax (Cheyer and Guzzoni, 2014; Langkilde and Knight, 1998). Though these frameworks can provide adequate information, their lack of nat- uralness and variation in language make the re- sponse rather rigid. Moreover, these methods usu- ally require non-trivial manual work to create tem- plates, rendering them unscalable across domains. Recently, corpus-based methods have gained considerable popularity in natural language gen- eration (Wen et al., 2015a,b; Duˇsek and Jurˇc´ıˇcek, 2016). With the increasing availability of rich di- alogue task data, corpus-based frameworks design end-to-end trainable systems. With minimum hu- man effort, these methods directly learn the pat- tern and styles of human responses from the data, while conveying the required task-specific mean- ing representation information. Furthermore, the booming of deep learning technology in natural language processing increases these models’ ca- pacity to generate sophisticated human-like re- sponses. For instance, Duˇsek and Jurˇc´ıˇcek (2016) employs sequence-to-sequence structure and at- tention mechanism to generate response tokens from the MR sequence. Wen et al. (2015b) uses …

Content truncated to 500 words