Modeling Context in Answer Sentence Selection Systems on a Latency Budget

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 3005–3010 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 3005 Modeling Context in Answer Sentence Selection Systems on a Latency Budget Rujun Han∗ University of Southern California Los Angeles, CA, USA rujunhan@usc.edu Luca Soldaini Amazon Alexa Manhattan Beach, CA, USA lssoldai@amazon.com Alessandro Moschitti Amazon Alexa Manhattan Beach, CA, USA amosch@amazon.com Abstract Answer Sentence Selection (AS2) is an effi- cient approach for the design of open-domain Question Answering (QA) systems. In or- der to achieve low latency, traditional AS2 models score question-answer pairs individu- ally, ignoring any information from the doc- ument each potential answer was extracted from. In contrast, more computationally ex- pensive models designed for machine reading comprehension tasks typically receive one or more passages as input, which often results in better accuracy. In this work, we present an approach to efficiently incorporate contextual information in AS2 models. For each answer candidate, we first use unsupervised similarity techniques to extract relevant sentences from its source document, which we then feed into an efficient transformer architecture fine-tuned for AS2. Our best approach, which leverages a multi-way attention architecture to efficiently encode context, improves 6% to 11% over non- contextual state of the art in AS2 with minimal impact on system latency. All experiments in this work were conducted in English. 1 Introduction AS2 models for open-domain QA typically con- sider sentences from webpages as independent candidate answers for a given question. For any webpage containing potential answer candidates for a question, AS2 models first extract individ- ual sentences, then independently estimate their likelihood of being correct answers; this approach enable highly efficient processing of entire docu- ments. However, under this framework, context information from the entire webpage (global con- text), which could be crucial for selecting correct answers, is ignored. Conversely, current systems in Machine Reading (MR) (Huang et al., 2019; ∗ Work was conducted while the author was an intern at Amazon Alexa. The math of pi explained, as simply as possible How many digits of pi we really need? Pi, you may also remember from grade school, is not an ordinary number. It’s irrational, meaning it has an endless number of decimals that never repeat. Though even cutting off pi at 15 digits allows for extremely precise measurements. If you were to draw a circle with a diameter of 25 billion miles, using 15 digits of pi, you’d only arrive at a measurement of the circumference that’s off by 1.5 inches, NASA’s Marc Rayman explained in a post on NASA’s JPL website. And that’s good enough. Of course, that hasn’t stopped people from looking for more and more digits of pi. Currently, there are more than 22.4 trillion known digits, which show no hint of ending or repeating. Further reading: pi and pie. Figure 1: Candidate sentence (in orange) for question “how many digits are in pi?”. Local context is shown in green, while global context is shown in red. …

Content truncated to 500 words