Multi-Grained Knowledge Retrieval for End-to-End Task-Oriented Dialog

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 11196–11210 July 9-14, 2023 ©2023 Association for Computational Linguistics Multi-Grained Knowledge Retrieval for End-to-End Task-Oriented Dialog Fanqi Wan1, Weizhou Shen1, Ke Yang1, Xiaojun Quan1∗, Wei Bi2∗ 1School of Computer Science and Engineering, Sun Yat-sen University, China 2Tencent AI Lab {wanfq, shenwzh3, yangk59}@mail2.sysu.edu.cn, quanxj3@mail.sysu.edu.cn, victoriabi@tencent.com Abstract Retrieving proper domain knowledge from an external database lies at the heart of end-to- end task-oriented dialog systems to generate informative responses. Most existing systems blend knowledge retrieval with response gen- eration and optimize them with direct supervi- sion from reference responses, leading to sub- optimal retrieval performance when the knowl- edge base becomes large-scale. To address this, we propose to decouple knowledge retrieval from response generation and introduce a multi- grained knowledge retriever (MAKER) that in- cludes an entity selector to search for relevant entities and an attribute selector to filter out irrelevant attributes. To train the retriever, we propose a novel distillation objective that de- rives supervision signals from the response gen- erator. Experiments conducted on three stan- dard benchmarks with both small and large- scale knowledge bases demonstrate that our retriever performs knowledge retrieval more ef- fectively than existing methods. Our code has been made publicly available.1 1 Introduction When task-oriented dialog (TOD) systems try to accomplish a task such as restaurant reservations and weather reporting for human users, they gen- erally resort to an external knowledge base (KB) to retrieve relevant entity information for generat- ing an informative system response. Conventional pipeline systems comprise several modules such as dialogue state tracking and dialogue policy learning that require annotations for training, where interme- diate predictions such as belief state can be used for the retrieval. By contrast, end-to-end task-oriented dialog (E2E-TOD) systems aim to eliminate the de- pendence on intermediate annotations and generate the response end-to-end (Wu et al., 2019). Appar- ently, knowledge retrieval is at the core of this task, ∗Corresponding authors 1https://github.com/18907305772/MAKER Condensed In-domain Cross-domain 28 30 32 34 36 38 Entity F1 CDNET FG2Seq EER DF-Net Figure 1: Performance of four end-to-end task-oriented dialog systems on MultiWOZ 2.1 when knowledge bases of different sizes are used. The evaluation metric is Entity F1 scores of entities in generated responses. “Condensed” means that each dialog is associated with a small-sized knowledge base, which is the default set- ting of many current systems. “In-domain” means that each dialog corresponds to a knowledge base of the same domain, while “Cross-domain” means that all di- alogs share the same large-scale cross-domain knowl- edge base provided in the dataset. which is non-trivial as no gold labels are available for training a retriever. Arguably, this problem has limited the performance of existing E2E-TOD sys- tems considering that substantial progress has been made in natural language generation. Roughly, existing approaches for knowledge re- trieval in E2E-TOD systems can be divided into three categories. First, the knowledge base can be embedded into a memory network and queried with the representations of …

Content truncated to 500 words