Context-Aware Answer Extraction in Question Answering

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2418–2428, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 2418 Context-Aware Answer Extraction in Question Answering Yeon Seonwoo†, Ji-Hoon Kim‡§, Jung-Woo Ha‡§, Alice Oh† †KAIST ‡NAVER AI LAB, §NAVER CLOVA yeon.seonwoo@kaist.ac.kr {genesis.kim,jungwoo.ha}@navercorp.com alice.oh@kaist.edu Abstract Extractive QA models have shown very promising performance in predicting the cor- rect answer to a question for a given passage. However, they sometimes result in predicting the correct answer text but in a context irrel- evant to the given question. This discrepancy becomes especially important as the number of occurrences of the answer text in a pas- sage increases. To resolve this issue, we pro- pose BLANC (BLock AttentioN for Context prediction) based on two main ideas: context prediction as an auxiliary task in multi-task learning manner, and a block attention method that learns the context prediction task. With experiments on reading comprehension, we show that BLANC outperforms the state-of- the-art QA models, and the performance gap increases as the number of answer text occur- rences increases. We also conduct an experi- ment of training the models using SQuAD and predicting the supporting facts on HotpotQA and show that BLANC outperforms all base- line models in this zero-shot setting. 1 Introduction Question answering tasks require a high level of reading comprehension ability, which in turn re- quires a high level of general language understand- ing. This is why the question answering (QA) tasks are often used to evaluate language models de- signed to be used in various language understand- ing tasks. Recent advances in contextual language models brought on by attention (Hermann et al., 2015; Chen et al., 2016; Seo et al., 2017; Tay et al., 2018) and transformers (Vaswani et al., 2017) have led to significant improvements in QA, and these improvements show that better modeling of contex- tual meanings of words plays a key role in QA. While these models are designed to select answer-spans in the relevant contexts from given passages, they sometimes result in predicting the Figure 1: Example passage, question, and answer triple. This passage has multiple spans that are matched with the answer text. The first occurrence of “prefrontal cor- tex” is the only answer-span within the context of the question. correct answer text but in contexts that are irrel- evant to the given questions. Figure 1 shows an example passage where the correct answer text appears multiple times. In this example, the only answer-span in the context relevant to the given question is the first occurrence of the “prefrontal cortex” (in blue), and all remaining occurrences of the answer text (in red) show incorrect predic- tions. Figure 2 shows quantitatively, the discrep- ancy between predicting the correct answer text versus predicting the correct answer-span. Using BERT (Devlin et al., 2019) trained on curated Nat- uralQuestions (Fisch et al., 2019), we show the results of extractive QA task using exact match (EM) and Span-EM. EM only looks for the text to match the ground truth answer, …

Content truncated to 500 words