DIALKI: Knowledge Identification in Conversational Systems through Dialogue-Document Contextualization

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1852–1863 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1852 DIALKI: Knowledge Identification in Conversational Systems through Dialogue-Document Contextualization Zeqiu Wu ♠∗ Bo-Ru Lu ♠∗ Hannaneh Hajishirzi ♠♦ Mari Ostendorf ♠ ♠University of Washington ♦Allen Institute for AI {zeqiuwu1,roylu,hannaneh,ostendor}@washington.edu Abstract Identifying relevant knowledge to be used in conversational systems that are grounded in long documents is critical to effective response generation. We introduce a knowledge iden- tification model that leverages the document structure to provide dialogue-contextualized passage encodings and better locate knowl- edge relevant to the conversation. An auxiliary loss captures the history of dialogue-document connections. We demonstrate the effective- ness of our model on two document-grounded conversational datasets and provide analyses showing generalization to unseen documents and long dialogue contexts. 1 Introduction Many conversational agent scenarios require knowledge-grounded response generation, where knowledge is represented in written documents. Most prior work has explored architectures for knowledge grounding in an end-to-end framework, optimizing a loss function targeting response gen- eration (Ghazvininejad et al., 2018; Zhou et al., 2018; Yavuz et al., 2019). However, the knowledge needed at any one turn in the dialogue is typically localized in the document, and some studies have shown that directly optimizing for knowledge ex- traction helps resolve complex user queries (Feng et al., 2020) and increases user engagement in so- cial chat (Dinan et al., 2019; Moghe et al., 2018). For long documents, explicit knowledge identifica- tion can also be useful for model interpretability and human-in-the-loop assistant scenarios. Following (Feng et al., 2020), we define knowl- edge identification as the task of locating knowl- edge in a long document that is relevant to the current user query given the conversation context (Figure 1). Knowledge identification is similar to open question answering (Chen et al., 2017; Min ∗Equal contribution [Sec 1] Private Service Bureau Licenses • Request Name Approval • Before you can apply for a license to operate a PSB, … • After your application is reviewed by the DMV in Albany, … • When the inspector visits your location, … [User]: Hi, can you tell me something about the private service bureau licenses? … [Agent]: Do you want to apply for a PSB? [User]: No, I was being curious. Just in case, what should I do if I apply for PSB? [Agent]: Your application will be reviewed in Albany's DMV. After that, it will be sent to your local DMV office and you'll be scheduled for an inspection. [Sec 3] How to Apply? [Sec 30] After you apply • A Private Service Bureau PSB license is required of … Grounding document Dialogue Context Figure 1: In a document-grounded conversation, knowl- edge identification targets to locate a knowledge string within a long document to assist the agent in addressing the current user query. et al., 2019), the task of answering a factoid ques- tion given a large grounding, except that it is not an interactive setting like dialogues. With the as- sumption …

Content truncated to 500 words