A Joint Model for Answer Sentence Ranking and Answer Extraction

A Joint Model for Answer Sentence Ranking and Answer Extraction Md Arafat Sultan† Vittorio Castelli‡ Radu Florian‡ †Institute of Cognitive Science and Department of Computer Science, University of Colorado, Boulder, CO ‡IBM T.J. Watson Research Center, Yorktown Heights, NY arafat.sultan@colorado.edu, vittorio@us.ibm.com, raduf@us.ibm.com Abstract Answer sentence ranking and answer extrac- tion are two key challenges in question answer- ing that have traditionally been treated in iso- lation, i.e., as independent tasks. In this arti- cle, we (1) explain how both tasks are related at their core by a common quantity, and (2) propose a simple and intuitive joint probabilis- tic model that addresses both via joint com- putation but task-specific application of that quantity. In our experiments with two TREC datasets, our joint model substantially outper- forms state-of-the-art systems in both tasks. 1 Introduction One of the original goals of AI was to build machines that can naturally interact with humans. Over time, the challenges became apparent and language pro- cessing emerged as one of AI’s most puzzling areas. Nevertheless, major breakthroughs have still been made in several important tasks; with IBM’s Wat- son (Ferrucci et al., 2010) significantly outperform- ing human champions in the quiz contest Jeopardy!, question answering (QA) is definitely one such task. QA comes in various forms, each supporting spe- cific kinds of user requirements. Consider a scenario where a system is given a question and a set of sen- tences each of which may or may not contain an answer to that question. The goal of answer extrac- tion is to extract a precise answer in the form of a short span of text in one or more of those sentences. In this form, QA meets users’ immediate information needs. Answer sentence ranking, on the other hand, is the task of assigning a rank to each sentence so that the ones that are more likely to contain an answer are ranked higher. In this form, QA is similar to informa- tion retrieval and presents greater opportunities for further exploration and learning. In this article, we propose a novel approach to jointly solving these two well-studied yet open QA problems. Most answer sentence ranking algorithms operate under the assumption that the degree of syntactic and/or semantic similarity between questions and an- swer sentences is a sufficiently strong predictor of answer sentence relevance (Wang et al., 2007; Yih et al., 2013; Yu et al., 2014; Severyn and Moschitti, 2015). On the other hand, answer extraction algo- rithms frequently assess candidate answer phrases based primarily on their own properties relative to the question (e.g., whether the question is a who question and the phrase refers to a person), making inadequate or no use of sentence-level evidence (Yao et al., 2013a; Severyn and Moschitti, 2013). Both these assumptions, however, are simplistic, and fail to capture the core requirements of the two tasks. Table 1 shows a question, and three candi- date answer sentences only one of which (S(1)) ac- tually answers the question. Ranking models that rely solely on text …

Content truncated to 500 words