Contextualize Knowledge Bases with Transformer for End-to-end Task-Oriented Dialogue Systems

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4300–4310 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4300 Contextualize Knowledge Bases with Transformer for End-to-end Task-Oriented Dialogue Systems Yanjie Gou1, Yinjie Lei1∗, Lingqiao Liu2, Yong Dai3, Chunxu Shen4 1College of Electronics and Information Engineering, Sichuan University, China 2School of Computer Science, The University of Adelaide, Australia 3University of Electronic Science and Technology of China, China 4Tencent gouyanjie@stu.scu.edu.cn, yinjie@scu.edu.cn, lingqiao.liu@adelaide.edu.au daiyongya@yahoo.com, lineshen@tencent.com Abstract Incorporating knowledge bases (KB) into end- to-end task-oriented dialogue systems is chal- lenging, since it requires to properly represent the entity of KB, which is associated with its KB context and dialogue context. The existing works represent the entity with only perceiv- ing a part of its KB context, which can lead to the less effective representation due to the information loss, and adversely favor KB rea- soning and response generation. To tackle this issue, we explore to fully contextualize the entity representation by dynamically perceiv- ing all the relevant entities and dialogue his- tory. To achieve this, we propose a COntext- aware Memory Enhanced Transformer frame- work (COMET), which treats the KB as a se- quence and leverages a novel Memory Mask to enforce the entity to only focus on its rele- vant entities and dialogue history, while avoid- ing the distraction from the irrelevant entities. Through extensive experiments, we show that our COMET framework can achieve superior performance over the state of the arts. 1 Introduction Task-oriented dialogue systems aim to achieve spe- cific goals such as hotel booking and restaurant reservation. The traditional pipelines (Young et al., 2013; Wen et al., 2017) consist of natural language understanding, dialogue management, and natural language generation modules. However, designing these modules often requires additional annotations such as dialogue states. To simplify this procedure, the end-to-end dialogue systems (Eric and Man- ning, 2017) are proposed to incorporate the KB (normally relational databases) into the learning framework, where the KB and dialogue history can be directly modeled for response generation, with- out the explicit dialogue state or dialogue action. ∗Corresponding author Poi Poi type Traffic Address Distance Stanford Express Care hospital moderate 214 El Camino Real 2 miles Tom’s house friend’s house no 580 Van Ness Ave 6 miles Philz coffee or tea place no 583 Alester Ave 4 miles 5672 Barringer Street certain address no 5672 Barringer Street 2 miles User Where does my friend live ? System Tom’s house is 6 miles away at 580 Van Ness Ave . User Is that the fastest route ? System I’ll send the route with no traffic on your screen , drive carefully ! Table 1: An example in SMD dataset (Eric et al., 2017). The top is the entities in KB and the bottom is a two- turn dialogue between the user and system. An example of the end-to-end dialogue systems is shown in Tab. 1. When generating the second response about the “traffic info”: (1) the targeted entity “no traffic” is associated with its …

Content truncated to 500 words