Conversational QA Dataset Generation with Answer Revision

Proceedings of the 29th International Conference on Computational Linguistics, pages 1636–1644 October 12–17, 2022. 1636 Conversational QA Dataset Generation with Answer Revision Seonjeong Hwang Graduate School of Artificial Intelligence POSTECH, Pohang, South Korea seonjeongh@postech.ac.kr Gary Geunbae Lee∗ Computer Science and Engineering Graduate School of Artificial Intelligence POSTECH, Pohang, South Korea gblee@postech.ac.kr Abstract Conversational question–answer generation is a task that automatically generates a large- scale conversational question answering dataset based on input passages. In this paper, we intro- duce a novel framework that extracts question- worthy phrases from a passage and then gener- ates corresponding questions considering previ- ous conversations. In particular, our framework revises the extracted answers after generating questions so that answers exactly match paired questions. Experimental results show that our simple answer revision approach leads to sig- nificant improvement in the quality of synthetic data. Moreover, we prove that our framework can be effectively utilized for domain adapta- tion of conversational question answering. 1 Introduction Conversational question answering (CQA) involves answering questions by considering a given text as well as previous conversations. To facilitate research on CQA, a range of datasets have been proposed in recent years (Choi et al., 2018; Reddy et al., 2019; Campos et al., 2020; Anantha et al., 2020; Adlakha et al., 2022). However, building a robust CQA system for a specific domain requires a large-scale domain-specific dataset; moreover, obtaining such a dataset is considerably expensive and time-consuming. To resolve this issue, in our previous study, we had proposed a conversational question–answer generation (CQAG) framework that automatically creates multiturn question–answer (Q–A) pairs from given passages (Hwang and Lee, 2021). The framework is a two-stage architecture that adopts contextual answer extraction (CAE) and conversa- tional question generation (CQG). Considering pre- vious conversations, the CAE module extracts the next question-worthy phrase from the passage, and then the CQG module generates the conversational ∗Corresponding author question corresponding to the phrase. However, the framework has the limitation that the error may propagate to the question generation stage and even to data generation for subsequent turns if improper answers are extracted by the CAE module. In this paper, we introduce a CQAG framework with answer revision (CQAG-AR), in which the conversational question generation with answer re- vision (CQG-AR) module revises the extracted an- swer to a more suitable one immediately after gen- erating a question. In experiments, we synthesize CQA data using CQAG-AR and then evaluate CQA models trained on these synthetic data. Results re- veal that answer revision by the CQG-AR module leads to absolute improvements of up to 13.4% and 15.3% in F1 score and exact match (EM), respec- tively, for the CQA models. Furthermore, fine- tuning the Wikipedia-domain CQA model on dif- ferent synthetic data increases EM by up to 13.1%, showing that our framework is beneficial for CQA domain adaptation. 2 Related Work CQG aims to create conversational questions based on input text. It can be subdivided into answer- aware and answer-unaware approaches. Answer- aware CQG generates conversational questions cor- responding to prepared answers (Gao …

Content truncated to 500 words