Disentangling Language and Knowledge in Task-Oriented Dialogs

Proceedings of NAACL-HLT 2019, pages 1239–1255 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1239 Disentangling Language and Knowledge in Task-Oriented Dialogs Dinesh Raghu∗1 2, Nikhil Gupta1 and Mausam1 1 IIT Delhi, New Delhi, India 2 IBM Research, New Delhi, India diraghu1@in.ibm.com, nikhilgupta1997@gmail.com, mausam@cse.iitd.ac.in Abstract The Knowledge Base (KB) used for real- world applications, such as booking a movie or restaurant reservation, keeps changing over time. End-to-end neural networks trained for these task-oriented dialogs are expected to be immune to any changes in the KB. However, existing approaches breakdown when asked to handle such changes. We propose an encoder- decoder architecture (BOSSNET) with a novel Bag-of-Sequences (BOSS) memory, which fa- cilitates the disentangled learning of the re- sponse’s language model and its knowledge incorporation. Consequently, the KB can be modified with new knowledge without a drop in interpretability. We find that BOSSNET out- performs state-of-the-art models, with consid- erable improvements (>10%) on bAbI OOV test sets and other human-human datasets. We also systematically modify existing datasets to measure disentanglement and show BOSSNET to be robust to KB modifications. 1 Introduction Task-oriented dialog agents converse with a user with the goal of accomplishing a specific task and often interact with a knowledge-base (KB). For example, a restaurant reservation agent (Henderson et al., 2014) will be grounded to a KB that contains the names of restaurants, and their details. In real-world applications, the KB information could change over time. For example, (1) a KB associated with a movie ticket booking system gets updated every week based on new film releases, and (2) a restaurant reservation agent, trained with the knowledge of eateries in one city, may be deployed in other cities with an entirely different range of es- tablishments. In such situations, the system should have the ability to conform to new-found knowl- edge unseen during its training. Ideally, the training algorithm must learn to disentangle the language ∗D. Raghu is an employee at IBM Research. This work was carried out as part of PhD research at IIT Delhi. Figure 1: Performance of various task-oriented dialog systems on the CamRest dataset as the percentage of unseen information in the KB changes. model from the knowledge interface model. This separation will enable the system to generalize to KB modifications, without a loss in performance. Moreover, for achieving good progress towards the user’s task, the agent must also retain the ability to draw inferences based on past utterances and the KB. Notably, we find that existing approaches either achieve this disentanglement or effective progress towards the task, but not both. For instance, Mem2Seq (Madotto et al., 2018) exhibits satisfactory performance when tested on the training KB. It represents the dialog history and the KB knowledge as a bag of words in a flat mem- ory arrangement. This enables Mem2Seq to revisit each word several times, as needed, obtaining good performance. But at the same time, flat memory prevents it from capturing any surrounding context – this deteriorates its performance …

Content truncated to 500 words