Constraint based Knowledge Base Distillation in End-to-End Task Oriented Dialogs

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 5051–5061 August 1–6, 2021. ©2021 Association for Computational Linguistics 5051 Constraint based Knowledge Base Distillation in End-to-End Task Oriented Dialogs Dinesh Raghu∗1 2, Atishya Jain1, Mausam1 and Sachindra Joshi2 1 IIT Delhi, New Delhi, India 2 IBM Research, New Delhi, India [diraghu1, jsachind]@in.ibm.com, [Atishya.Jain.cs516, mausam]@cse.iitd.ac.in Abstract End-to-End task-oriented dialogue systems generate responses based on dialog history and an accompanying knowledge base (KB). Infer- ring those KB entities that are most relevant for an utterance is crucial for response gener- ation. Existing state of the art scales to large KBs by softly filtering over irrelevant KB in- formation. In this paper, we propose a novel filtering technique that consists of (1) a pair- wise similarity based filter that identifies rele- vant information by respecting the n-ary struc- ture in a KB record. and, (2) an auxiliary loss that helps in separating contextually unrelated KB information. We also propose a new met- ric – multiset entity F1 which fixes a correct- ness issue in the existing entity F1 metric. Ex- perimental results on three publicly available task-oriented dialog datasets show that our pro- posed approach outperforms existing state-of- the-art models. 1 Introduction Task oriented dialog systems interact with users to achieve specific goals such as restaurant reservation or calendar enquiry. To satisfy a user goal, the sys- tem is expected to retrieve necessary information from a knowledge base and convey it using natural language. Recently several end-to-end approaches (Bordes and Weston, 2017; Wu et al., 2018; He et al., 2020b; Madotto et al., 2018) have been pro- posed for learning these dialog systems. Inferring the most relevant KB entities necessary for generating the response is crucial for achiev- ing task success. To effectively scale to large KBs, existing approaches (Wen et al., 2018; Wu et al., 2018) distill the KB by softly filtering irrelevant KB information based on the dialog history. For example, in Figure 1 the ideal filtering technique ∗D. Raghu is an employee at IBM Research. This work was carried out as part of PhD research at IIT Delhi. When is my dinner with Alex, the date and time? The dinner with Alex is at 10am on 1st Feb. driver # Date Event Invitee Time 1 1st Feb dinner Alex 10am 2 7th Feb dentist appointment - 2pm 3 8th Feb dinner Ana 7pm system Figure 1: An example dialog between a driver and a system along with the associated knowledge base. is expected to filter just the row 1 as the driver is requesting information about dinner with Alex. But existing techniques often filter some irrelevant KB information along with the relevant KB informa- tion. For example, in Figure 1 row 3 may also get filtered along with row 1. Our analysis of the best performing distillation technique (Wu et al., 2018) revealed that embed- dings learnt for entities of the same type are quite close to each other. This may be due to entities of the same type …

Content truncated to 500 words