Semi-Supervised Bootstrapping of Dialogue State Trackers for Task-Oriented Modelling

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1273–1278, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1273 Semi-Supervised Bootstrapping of Dialogue State Trackers for Task-Oriented Modelling Bo-Hsiang Tseng1, Marek Rei2,4, Paweł Budzianowski1,3,5 Richard E. Turner1, Bill Byrne1, Anna Korhonen3 1Engineering Department, Cambridge University, UK 2Computer Laboratory, Cambridge University, UK 3Language Technology Lab, Cambridge University, UK 4Department of Computing, Imperial College London 5PolyAI Limited, London, UK bht26@cam.ac.uk Abstract Dialogue systems benefit greatly from opti- mizing on detailed annotations, such as tran- scribed utterances, internal dialogue state rep- resentations and dialogue act labels. However, collecting these annotations is expensive and time-consuming, holding back development in the area of dialogue modelling. In this paper, we investigate semi-supervised learning meth- ods that are able to reduce the amount of re- quired intermediate labelling. We find that by leveraging un-annotated data instead, the amount of turn-level annotations of dialogue state can be significantly reduced when build- ing a neural dialogue system. Our analysis on the MultiWOZ corpus, covering a range of domains and topics, finds that annotations can be reduced by up to 30% while maintain- ing equivalent system performance. We also describe and evaluate the first end-to-end di- alogue model created for the MultiWOZ cor- pus. 1 Introduction Task-oriented dialogue models aim at assisting with well-defined and structured problems like booking tickets or providing information to visi- tors in a new city (Raux et al., 2005). Most cur- rent industry-oriented systems rely on modular, domain-focused frameworks (Young et al., 2013; Sarikaya et al., 2016), with separate components for user understanding (Henderson et al., 2014), decision making (Gaši´c et al., 2010) and system answer generation (Wen et al., 2015). Recent progress in sequence-to-sequence (seq2seq) mod- elling has enabled the development of fully neu- ral end-to-end architectures, allowing for differ- ent components to be optimized jointly in order to share information (Wen et al., 2017; Zhao et al., 2017; Budzianowski and Vuli´c, 2019). Dialogue systems benefit greatly from optimiz- ing on detailed annotations, such as turn-level dia- logue state labels (Henderson et al., 2014) or di- alogue actions (Rieser and Lemon, 2011), with end-to-end architectures still relying on interme- diate labels in order to obtain satisfactory results (Lei et al., 2018). Collecting these labels is often the bottleneck in dataset creation, as the process is expensive and time-consuming, requiring domain and expert knowledge (Asri et al., 2017). Due to this restriction, existing datasets for task-oriented dialogue are several orders of magnitude smaller compared to general open-domain dialogue cor- pora (Lowe et al., 2015; Henderson et al., 2019). Arguably, one of the most challenging parts of dialogue modelling is maintaining an interpretable internal memory over crucial domain concepts (Williams et al., 2016). Although there is increas- ing research effort to learn Dialogue State Track- ing (DST) jointly with the text generation compo- nent (Eric et al., 2017; Wu et al., 2019), the most effective models use it as …

Content truncated to 500 words