Answer Generation for Retrieval-based Question Answering Systems

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4276–4282 August 1–6, 2021. ©2021 Association for Computational Linguistics 4276 Answer Generation for Retrieval-based Question Answering Systems Chao-Chun Hsu1∗, Eric Lind2, Luca Soldaini2, Alessandro Moschitti2 1University of Chicago, 2Amazon Alexa chaochunh@uchicago.edu, {lssoldai,ericlind,amosch}@amazon.com Abstract Recent advancements in transformer-based models have greatly improved the ability of Question Answering (QA) systems to provide correct answers; in particular, answer sentence selection (AS2) models, core components of retrieval-based systems, have achieved impres- sive results. While generally effective, these models fail to provide a satisfying answer when all retrieved candidates are of poor quality, even if they contain correct information. In AS2, models are trained to select the best answer sentence among a set of candidates retrieved for a given question. In this work, we propose to generate answers from a set of AS2 top candi- dates. Rather than selecting the best candidate, we train a sequence to sequence transformer model to generate an answer from a candidate set. Our tests on three English AS2 datasets show improvement up to 32 absolute points in accuracy over the state of the art. 1 Introduction Question answering systems are a core component of many commercial applications, ranging from task-based dialog systems to general purpose vir- tual assistants, e.g., Google Home, Amazon Alexa, and Siri. Among the many approaches for QA, AS2 has attracted significant attention in the last few years (Tymoshenko and Moschitti, 2018; Tian et al., 2020; Garg et al., 2020; Han et al., 2021). Under this framework, for a given question, a re- trieval system is first used to obtain and rank a set of supporting passages; then, an AS2 model is used to estimate the likelihood of each sentence extracted from passages to be a correct answer, re- turning the one with the highest probability. This approach is favored in virtual assistant systems be- cause full sentences are more likely to include the ∗ This work was completed while the author was an intern at Amazon Alexa. Q: How a water pump works? c1: A small, electrically powered pump. c2: A large, electrically driven pump (electropump) for wa- terworks near the Hengsteysee, Germany. c3: A pump is a device that moves fluids (liquids or gases), or sometimes slurries, by mechanical action. c4: Pumps can be classified into three major groups according to the method they use to move the fluid: direct lift, displacement, and gravity pumps. c5: Pumps operate by some mechanism (typically reciprocat- ing or rotary), and consume energy to perform mechanical work by moving the fluid. G: A water pump is a device that moves fluids by mechani- cal action. Table 1: An example of a question Q and five answer candidates c1, . . . , c5 from WikiQA (Yang et al., 2015) ranked by an AS2 system. Answer G generated by our best system is significantly more natural and concise than any extracted candidates. right context and sound natural, both of which are characteristics users value (Berdasco et al., 2019). AS2 …

Content truncated to 500 words