Joint Models for Answer Verification in Question Answering Systems

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 3252–3262 August 1–6, 2021. ©2021 Association for Computational Linguistics 3252 Joint Models for Answer Verification in Question Answering Systems Zeyu Zhang∗ Thuy Vu Alessandro Moschitti School of Information, The University of Arizona, Tucson, AZ, USA Amazon Alexa AI, Manhattan Beach, CA, USA zeyuzhang@email.arizona.edu, {thuyvu, amosch}@amazon.com Abstract This paper studies joint models for selecting correct answer sentences among the top k pro- vided by answer sentence selection (AS2) mod- ules, which are core components of retrieval- based Question Answering (QA) systems. Our work shows that a critical step to effectively ex- ploiting an answer set regards modeling the in- terrelated information between pair of answers. For this purpose, we build a three-way multi- classifier, which decides if an answer supports, refutes, or is neutral with respect to another one. More specifically, our neural architecture integrates a state-of-the-art AS2 module with the multi-classifier, and a joint layer connecting all components. We tested our models on Wik- iQA, TREC-QA, and a real-world dataset. The results show that our models obtain the new state of the art in AS2. 1 Introduction Automated Question Answering (QA) research has received a renewed attention thanks to the diffusion of Virtual Assistants. Among the different types of methods to implement QA systems, we focus on Answer Sentence Selection (AS2) research, orig- inated from TREC-QA track (Voorhees and Tice, 1999), as it proposes efficient models that are more suitable for a production setting, e.g., they are more efficient than those developed in machine reading (MR) work (Chen et al., 2017). Garg et al. (2020) proposed the TANDA ap- proach based on pre-trained Transformer models, obtaining impressive improvement over the state of the art for AS2, measured on the two most used datasets, WikiQA (Yang et al., 2015) and TREC- QA (Wang et al., 2007). However, TANDA was applied only to pointwise rerankers (PR), e.g., sim- ple binary classifiers. Bonadiman and Moschitti ∗Work done while the author was an intern at Amazon Alexa Claim: Joe Walsh was inducted in 2001. Ev1: As a member of the Eagles, Walsh was inducted into the Rock and Roll Hall of Fame in 1998, and into the Vocal Group Hall of Fame in 2001. Ev2: Joseph Fidler Walsh (born November 20, 1947) is an American singer songwriter, composer, multi- instrumentalist and record producer. Ev3: Walsh was awarded with the Vocal Group Hall of Fame in 2001. Table 1: A claim verification example from FEVER. (2020) tried to improve this model by jointly mod- eling all answer candidates with listwise methods, e.g., (Bian et al., 2017). Unfortunately, merging the embeddings from all candidates with standard approaches, e.g., CNN or LSTM, did not improve over TANDA. A more structured approach to building joint models over sentences can instead be observed in Fact Verification Systems, e.g., the methods de- veloped in the FEVER challenge (Thorne et al., 2018a). Such systems take a claim, e.g., …

Content truncated to 500 words