Double Retrieval and Ranking for Accurate Question Answering

Findings of the Association for Computational Linguistics: EACL 2023, pages 1751–1762 May 2-6, 2023 ©2023 Association for Computational Linguistics Double Retrieval and Ranking for Accurate Question Answering Zeyu Zhang1∗, Thuy Vu2, and Alessandro Moschitti2 1School of Information, The University of Arizona, Tucson, AZ, USA 2Amazon Alexa AI, Manhattan Beach, CA, USA zeyuzhang@arizona.edu, {thuyvu, amosch}@amazon.com Abstract Recent work has shown that an answer veri- fication step introduced in Transformer-based answer selection models can significantly im- prove the state of the art in Question Answering. This step is performed by aggregating the em- beddings of top k answer candidates to support the verification of a target answer. Although the approach is intuitive and sound, it still shows two limitations: (i) the supporting candidates are ranked only according to the relevancy with the question and not with the answer, and (ii) the support provided by the other answer can- didates is suboptimal as these are retrieved in- dependently of the target answer. In this paper, we address both drawbacks by proposing (i) a double reranking model, which, for each tar- get answer, selects the best support; and (ii) a second neural retrieval stage designed to en- code question and answer pair as the query, which finds more specific verification informa- tion. The results on well-known datasets for Answer Sentence Selection show significant improvement over the state of the art. 1 Introduction In recent years, automated Question Answering (QA) research has received a renewed attention thanks to the diffusion of Virtual Assistants. For example, Google Home, Siri and Alexa provide general information inquiry services, while many other systems serve customer requests in differ- ent application domains. Retrieval-based QA is enabled by two main tasks: (i) Answer Sentence Selection (AS2), which, given a question and a set of answer-sentence candidates, consists in selecting sentences (e.g., retrieved by a search engine) that correctly answer the question; and (ii) Machine Reading (MR), e.g., (Chen et al., 2017), which, given a question and a reference text, finds an exact text span that answers the question. Deploying MR ∗ Work done while the author was an intern at Amazon Alexa q: What causes heart disease? c1: Cardiovascular disease (also called heart disease) is a class of diseases that involve the heart or blood vessels (arteries, capillaries, and veins). c2: The causes of cardiovascular disease are diverse but atherosclerosis and/or hypertension are the most com- mon. c3: Cardiovascular disease refers to any disease that affects the cardiovascular system , principally cardiac disease, vascular diseases of the brain and kidney , and peripheral arterial disease. Table 1: A question with answer candidates. systems in production is challenging for efficiency reasons, while AS2 models can efficiently target large text databases. Indeed, they originated from TREC QA tracks (Voorhees and Tice, 1999), which dealt with real-world retrieval systems since the first edition. Another limitation of MR is the focus on factoid answers: although it can in principle pro- vide longer answers, the datasets developed for the task mainly contains short answers and in …

Content truncated to 500 words