monoQA: Multi-Task Learning of Reranking and Answer Extraction for Open-Retrieval Conversational Question Answering

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7207 - 7218 December 7-11, 2022 ©2022 Association for Computational Linguistics monoQA: Multi-Task Learning of Reranking and Answer Extraction for Open-Retrieval Conversational Question Answering Sarawoot Kongyoung1, Craig Macdonald2, Iadh Ounis2 University of Glasgow, UK 1s.kongyoung.1@research.gla.ac.uk 2{craig.macdonald,iadh.ounis}@glasgow.ac.uk Abstract To address the Conversational Question An- swering (ORConvQA) task, previous work has considered an effective three-stage architec- ture, consisting of a retriever, a reranker, and a reader to extract the answers. In order to ef- fectively answer the users’ questions, a number of existing approaches have applied multi-task learning, such that the same model is shared between the reranker and the reader. Such ap- proaches also typically tackle reranking and reading as classification tasks. On the other hand, recent text generation models, such as monoT5 and UnifiedQA, have been shown to respectively yield impressive performances in passage reranking and reading. However, no prior work has combined monoT5 and Uni- fiedQA to share a single text generation model that directly extracts the answers for the users instead of predicting the start/end positions in a retrieved passage. In this paper, we investi- gate the use of Multi-Task Learning (MTL) to improve performance on the ORConvQA task by sharing the reranker and reader’s learned structure in a generative model. In particular, we propose monoQA, which uses a text genera- tion model with multi-task learning for both the reranker and reader. Our model, which is based on the T5 text generation model, is fine-tuned simultaneously for both reranking (in order to improve the precision of the top retrieved pas- sages) and extracting the answer. Our results on the OR-QuAC and OR-CoQA datasets demon- strate the effectiveness of our proposed model, which significantly outperforms existing strong baselines with improvements ranging from +12.31% to +19.51% in MAP and from +5.70% to +23.34% in F1 on all used test sets. 1 Introduction Research in Conversational Search and Conver- sational Question Answering (ConvQA) is of increasing interest. The ConvQA task (Choi et al., 2018; Reddy et al., 2019) consists in understanding Figure 1: Overview of (a) reranker and extractive reader and (b) reranker and generative reader. the question based on a given conversational his- tory, and extracting an answer from a given passage. This task is an extractive type of QA, meaning that the answer takes the form of a span in the provided passage, and can be successfully tackled by employing an extractive or generative reader. Extractive reader models (Qu et al., 2019a,b; Yeh and Chen, 2019) are typically employed, where the goal is to classify the start and end positions of the answer span in the given passage. In contrast, generative readers (Raffel et al., 2020; Khashabi et al., 2020; Lewis et al., 2020; Karpukhin et al., 2020) have demonstrated impressive results on the extractive QA tasks, where the goal is to generate tokens that are a subset of a passage. Recently, there has been more focus on retrieval as part of the ConvQA pipeline, known …

Content truncated to 500 words