A Generative Model for Joint Natural Language Understanding and Generation

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1795–1807 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 1795 A Generative Model for Joint Natural Language Understanding and Generation Bo-Hsiang Tseng1∗, Jianpeng Cheng2, Yimai Fang2 and David Vandyke2 1Engineering Department, University of Cambridge, UK 2Apple bht26@cam.ac.uk {jianpeng.cheng, yimai_fang, dvandyke}@apple.com Abstract Natural language understanding (NLU) and natural language generation (NLG) are two fundamental and related tasks in building task-oriented dialogue systems with opposite objectives: NLU tackles the transformation from natural language to formal representa- tions, whereas NLG does the reverse. A key to success in either task is parallel training data which is expensive to obtain at a large scale. In this work, we propose a gener- ative model which couples NLU and NLG through a shared latent variable. This ap- proach allows us to explore both spaces of natural language and formal representations, and facilitates information sharing through the latent space to eventually benefit NLU and NLG. Our model achieves state-of-the-art per- formance on two dialogue datasets with both flat and tree-structured formal representations. We also show that the model can be trained in a semi-supervised fashion by utilising unla- belled data to boost its performance. 1 Introduction Natural language understanding (NLU) and natural language generation (NLG) are two fundamental tasks in building task-oriented dialogue systems. In a modern dialogue system, an NLU module first converts a user utterance, provided by an automatic speech recognition model, into a formal represen- tation. The representation is then consumed by a downstream dialogue state tracker to update a be- lief state which represents an aggregated user goal. Based on the current belief state, a policy network decides the formal representation of the system re- sponse. This is finally used by an NLG module to generate the system response(Young et al., 2010). It can be observed that NLU and NLG have op- posite goals: NLU aims to map natural language ∗Work done while the author was an intern at Apple. Figure 1: Generation and inference process in our model, and how NLU and NLG are achieved. x and y denotes utterances and formal representations respec- tively; z represents the shared latent variable for x and y. to formal representations, while NLG generates utterances from their semantics. In research liter- ature, NLU and NLG are well-studied as separate problems. State-of-the-art NLU systems tackle the task as classification (Zhang and Wang, 2016) or as structured prediction or generation (Damonte et al., 2019), depending on the formal representa- tions which can be flat slot-value pairs (Henderson et al., 2014), first-order logical form (Zettlemoyer and Collins, 2012), or structured queries (Yu et al., 2018; Pasupat et al., 2019). On the other hand, approaches to NLG vary from pipelined approach subsuming content planning and surface realisation (Stent et al., 2004) to more recent end-to-end se- quence generation (Wen et al., 2015; Dušek et al., 2020). However, the duality between NLU and NLG has been less explored. In fact, both tasks can be treated …

Content truncated to 500 words