Complementary Evidence Identification in Open-Domain Question Answering

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2720–2726 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2720 Complementary Evidence Identification in Open-Domain Question Answering Xiangyang Mou Rensselaer Polytechnic Institute moux4@rpi.edu Mo Yu IBM Research yum@us.ibm.com Shiyu Chang MIT-IBM Watson AI Lab shiyu.chang@ibm.com Yufei Feng Queen’s University feng.yufei@queensu.ca Li Zhang Amazon Web Services lzhangza@amazon.com Hui Su Fidelity Hui.Su@fmr.com Abstract This paper proposes a new problem of com- plementary evidence identification for open- domain question answering (QA). The prob- lem aims to efficiently find a small set of pas- sages that covers full evidence from multiple aspects as to answer a complex question. To this end, we proposes a method that learns vector representations of passages and mod- els the sufficiency and diversity within the se- lected set, in addition to the relevance between the question and passages. Our experiments demonstrate that our method considers the de- pendence within the supporting evidence and significantly improves the accuracy of comple- mentary evidence selection in QA domain. 1 Introduction In recent years, significant progress has been made in the field of open-domain question answering (Chen et al., 2017; Wang et al., 2017, 2018; Clark and Gardner, 2018; Min et al., 2018; Asai et al., 2019). Very recently, some works turn to deal with a more challenging task of asking complex ques- tions (Welbl et al., 2018; Clark et al., 2018; Yang et al., 2018) from the open-domain text corpus. In the open-domain scenario, one critical challenge raised by complex questions is that each question may require multiple pieces of evidence to get the right answer, while the evidence usually scatters in different passages. Examples in Figure 1 shows two types of questions that require evidence from multiple passages. To deal with the challenging multi-evidence questions, an open-domain QA system should be able to (1) efficiently retrieve a small number of passages that cover the full evidence; and (2) ac- curately extract the answer by jointly consider- ing the candidate evidence passages. While there have been several prior works in the latter direc- tion (Wang et al., 2017; Clark and Gardner, 2018; Figure 1: Examples of complex questions involving two facts of a person. Different facts are color-coded. P# are all relevant passages, while only the ones with solid-line boxes are the true supporting passages. Lin et al., 2018), the solutions to the first problem still rely on traditional or neural information re- trieval (IR) approaches, which solely measure the relevance between the question and each individual paragraph, and will highly possibly put the wrong evidence to the top.1 For example in Figure 1 (top), P1 and P2 are two candidate evidence passages that are closely related to the question but only cover the same unilateral fact required by the question, therefore leading us to the wrong answer Newton. This paper formulates a new problem of com- plementary evidence identification for answer- ing complex questions. The key idea is to consider the problem as …

Content truncated to 500 words