Generating Information-Seeking Conversations from Unlabeled Documents

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2362 - 2378 December 7-11, 2022 ©2022 Association for Computational Linguistics Generating Information-Seeking Conversations from Unlabeled Documents Gangwoo Kim1♣∗ Sungdong Kim2,3∗ Kang Min Yoo2,4 Jaewoo Kang1† Korea University1 NAVER AI Lab2 KAIST AI3 NAVER CLOVA4 {gangwoo_kim, kangj}@korea.ac.kr {sungdong.kim, kangmin.yoo}@navercorp.com Abstract Synthesizing datasets for conversational ques- tion answering (CQA) from unlabeled docu- ments remains challenging due to its interac- tive nature. Moreover, while modeling informa- tion needs is an essential key, only few studies have discussed it. In this paper, we introduce a novel framework, SIMSEEK, (Simulating information-Seeking conversation from unla- beled documents), and compare its two variants. In our baseline SIMSEEK-SYM, a questioner generates follow-up questions upon the prede- termined answer by an answerer. On the con- trary, SIMSEEK-ASYM first generates the ques- tion and then finds its corresponding answer under the conversational context. Our experi- ments show that they can synthesize effective training resources for CQA and conversational search tasks. As a result, conversations from SIMSEEK-ASYM not only make more improve- ments in our experiments but also are favorably reviewed in a human evaluation. We finally re- lease a large-scale resource of synthetic conver- sations, WIKI-SIMSEEK, containing 2 million CQA pairs built upon Wikipedia documents. With the dataset, our CQA model achieves the state-of-the-art performance on a recent CQA benchmark, QuAC (Choi et al., 2018) 1. 1 Introduction Conversational question answering (CQA) involves modeling the information-seeking process of hu- man dialogue. In the task, systems should under- stand questions according to the conversational con- text. To build robust systems, large-scale CQA datasets (Choi et al., 2018; Reddy et al., 2019; Saeidi et al., 2018; Campos et al., 2020) have re- cently been introduced. Still, they are limited in scale to generalize toward real-world applications, ♣ Work done while interning at NAVER AI Lab ∗ Equal contribution † Corresponding author 1The code and dataset are available at github.com/naver- ai/simseek. Juanita Moore a1 Who is the 3rd African American actress to get a nomination for supporting actress Oscar? q 1 Evidence Document Natural causes a2 How did she pass away? q 2 Topic : Lionel Messi Section : 2012 - FC Barcelona Background Evidence Document Juanita Moore, the third African- American actress to ever get a supporting actress Oscar nomination, died of natural causes … CANNOTANSWER (Not in the document) a2 Did the team do well this year? q 1 Who was their coach? q 2 (a) Information-symmetric Conversation (b) Information-asymmetric Conversation the team was eliminated in the semi-final a1 Figure 1: Examples of two conversation scenarios. In the former, questioners with access to the evidence doc- ument often ask less related things to the conversation. In the latter, questioners seek new information from the inaccessible document, leading to information-seeking behaviors, i.e., open-ended and unanswerable questions. which motivates the development of automated methods for constructing CQA datasets. However, generating CQA datasets is a challeng- ing task, which requires interactions between ques- tioner and answer. Therefore, most of the …

Content truncated to 500 words