MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6162–6176 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6162 MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents Song Feng∗ IBM Research AI sfeng@us.ibm.com Siva Sankalp Patel∗ IBM Research AI siva.sankalp.patel@ibm.com Hui Wan IBM Research AI hwan@us.ibm.com Sachindra Joshi IBM Research AI jsachind@in.ibm.com Abstract We propose MultiDoc2Dial, a new task and dataset on modeling goal-oriented dialogues grounded in multiple documents. Most previ- ous works treat document-grounded dialogue modeling as a machine reading comprehen- sion task based on a single given document or passage. In this work, we aim to ad- dress more realistic scenarios where a goal- oriented information-seeking conversation in- volves multiple topics, and hence is grounded on different documents. To facilitate such a task, we introduce a new dataset that contains dialogues grounded in multiple documents from four different domains. We also explore modeling the dialogue-based and document- based context in the dataset. We present strong baseline approaches and various experimental results, aiming to support further research ef- forts on such a task. 1 Introduction With the recent advancements in NLP, there has been a surge of research interests and efforts in developing conversational systems for various do- mains. An important task in the field is conversa- tional question answering and document-grounded dialogue modeling. Prior work typically formu- lates the task as a machine reading comprehension task assuming the associated document or text snip- pet is given, such as QuAC (Choi et al., 2018), ShARC (Saeidi et al., 2018), CoQA (Reddy et al., 2019), OR-QuAC (Qu et al., 2020a) and Doc2Dial (Feng et al., 2020b). However, such task setup ne- glects the common real-life scenarios where a goal- oriented conversation could correspond to several sub-goals that are addressed in different documents. In this work, we propose a new task and dataset, MultiDoc2Dial, on modeling goal-oriented dia- logues that are grounded in multiple documents. We illustrate the proposed task in Figure 1. It in- cludes a goal-oriented dialogue with four segments * Equal contribution. on the left and three relevant documents on the right. Each dialogue segment indicates that all turns within it are grounded in a same document, e.g., turns from A3 to A7 in Seg-2 are all grounded in Doc-2. The blue dashed lines connect a dialogue turn with its corresponding relevant span in a doc- ument. The red dotted lines with arrows indicate that the dialogue flow shifts among the grounding documents through the conversation, i.e., Doc-1 → Doc-2 → Doc-1 → Doc-3. This example highlights certain challenges for identifying the relevant grounding content among different docu- ments dynamically in a conversation. For instance, agent response A2 mentions ‘insured’ as an impor- tant condition based on a span in Doc-1. However, there are no more details about ‘insured’ in Doc-1. To further discuss about ‘insured‘, the conversation naturally switches to refer to another document Doc-2. Another challenge is to handle deep dia- logue context. For instance, to provide response to U4 or U6 in …

Content truncated to 500 words