Retrieval-free Knowledge Injection through Multi-Document Traversal for Dialogue Models

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6608–6619 July 9-14, 2023 ©2023 Association for Computational Linguistics Retrieval-free Knowledge Injection through Multi-Document Traversal for Dialogue Models Abstract Dialogue models are often enriched with ex- tensive external knowledge to provide informa- tive responses through a retrieval-augmented pipeline. Nevertheless, retrieval-augmented ap- proaches rely on finely annotated retrieval train- ing data and knowledge-grounded response generation data, making it costly to transfer. To tackle this challenge, this paper proposed a retrieval-free approach, KiDG, by automati- cally turning knowledge documents into sim- ulated multi-turn dialogues through a Multi- Document Traversal algorithm. The simulated knowledge-intensive dialogues constructed by KiDG in one domain can be easily used to train and enhance pre-trained dialogue models’ knowledge w.r.t. this domain without costly annotation. We conduct extensive experiments comparing retrieval-augmented models and a variety of retrieval-free models. We found that dialogue models enhanced with data sim- ulated with KiDG largely outperform state-of- the-art retrieval-free methods, and it achieves comparable performance compared to retrieval- augmented methods while being better, and cheaper at domain transfer. We have released the code and data at https://github.com/ DevoAllen/KiDG. 1 Introduction Knowledge plays a crucial role in dialogue systems, which is helpful in improving the informativeness, logicality, and reliability of generated responses. To encourage Pretrained Dialogue Models (PDMs) to produce knowledge-grounded responses, exist- ing research mainly follows two lines: retrieval- based (Dinan et al., 2019; Zhao et al., 2020) and retrieval-free(Xu et al., 2022). ∗ This work was done during the internship at Huawei Noah’s Ark Lab. †Corresponding Authors. Taylor Swift Taylor released the album "1989" in 2014. In 2010, Taylor released "Speak Now". In 2022, she released her music album "Midnights". Taylor Swift was born on December 13, 1989. She is an American singer. She is also an actress. In 2010, she received 8 nominations for the 52nd Grammy Awards. In 2014 she won the 58th Grammy Award for Album of the Year. Topic Aspect Awards Album Others Speak Now Awards "Speak Now" debuted at No. 1 on the Billboard 200 album chart. released released by Figure 1: The structure of the document "Taylor Swift" and her album "Speak Now". A document usually con- centrates on one topic. Sentences within the document describe different aspects of the document’s topic. The retrieval-based paradigm explicitly provides DMs with ready-to-use knowledge by a knowledge retriever (Karpukhin et al., 2020) and trains DMs to apply it during response generation on knowledge- grounded conversations (Izacard and Grave, 2021). Yet, it relies on finely annotated retrieval train- ing data as well as knowledge-grounded conver- sations, making it costly to apply and transfer. The retrieval-free paradigm aims to directly incorporate knowledge into DMs through training on simulated knowledge-intensive conversational data. There- fore, it’s more convenient and cheaper to be applied at scale. Recently, Dai et al. (2022) proposed dialogue inpainting to automatically transform a single doc- ument into a multi-turn dialogue. However, di- alogue data produced by the existing inpainting approach only considers …

Content truncated to 500 words