doc2dial: A Goal-Oriented Document-Grounded Dialogue Dataset

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 8118–8128, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 8118 doc2dial: A Goal-Oriented Document-Grounded Dialogue Dataset Song Feng Hui Wan Chulaka Gunasekara Siva Sankalp Patel Sachindra Joshi Luis A. Lastras IBM Research AI {sfeng@us, hwan@us, chulaka.gunasekara@}.ibm.com {siva.sankalp.patel@, jsachind@in, lastrasl@us}.ibm.com Abstract We introduce doc2dial, a new dataset of goal-oriented dialogues that are grounded in the associated documents. Inspired by how the authors compose documents for guiding end users, we first construct dialogue flows based on the content elements that corresponds to higher-level relations across text sections as well as lower-level relations between dis- course units within a section. Then we present these dialogue flows to crowd contributors to create conversational utterances. The dataset includes over 4500 annotated conversations with an average of 14 turns that are grounded in over 450 documents from four domains. Compared to the prior document-grounded di- alogue datasets, this dataset covers a variety of dialogue scenes in information-seeking con- versations. For evaluating the versatility of the dataset, we introduce multiple dialogue model- ing tasks and present baseline approaches. 1 Introduction The task of reading documents and responding to queries has been the trigger of many recent research advances. On top of the development of contex- tual question answering QuAC (Choi et al., 2018) and CoQA (Reddy et al., 2019), more recent work MANtIS (Penha et al., 2019) and DoQA (Campos et al., 2020) included more kinds of user intents for querying over documents; while ShARC (Saeidi et al., 2018) added follow-up questions from agents and binary answers from users for the inference over documents. These exciting works confirm the importance of modeling document-grounded dialogue. Yet, it involves more complex scenes in practice, which requires better understanding of the inter-relations between conversations and documents. Thus, we aim to investigate how to create the training instances to further approach real-world applications of document-grounded dia- logue for information seeking tasks. In this work, we propose a new dataset of goal- oriented document-grounded dialogue. Figure 1 shows sample utterances from dialogues D1, D2 and D3 between an assisting agent and a user, and an example document in the middle. D1 and D2 are grounded in the given document, while D3 is irrelevant to the document. It illustrates two dif- ferent types of contexts that we aim to capture: (1) dialogue-based context, where a query could be formed by a single or multiple turns, and (2) document-based context, which corresponds to var- ied forms of knowledge represented in the docu- ment. More specifically, dialogue-based context of a query could be initiated by a user (e.g., U1 in D1) or an agent (e.g., A3 in D1), and carried out through multiple turns by both roles (e.g., all turns in D2). Document-based context could in- volve structural elements in documents, such as the headers T1 and T2 or list items of m1 and m2, as well as textual discourse units, such as clauses (e.g, “If your clothing has been damaged”). For …

Content truncated to 500 words