Question Answering Using Enhanced Lexical Semantic Models

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 1744–1753, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Question Answering Using Enhanced Lexical Semantic Models Wen-tau Yih Ming-Wei Chang Christopher Meek Andrzej Pastusiak Microsoft Research Redmond, WA 98052, USA {scottyih,minchang,meek,andrzejp}@microsoft.com Abstract In this paper, we study the answer sentence selection problem for ques- tion answering. Unlike previous work, which primarily leverages syntactic analy- sis through dependency tree matching, we focus on improving the performance us- ing models of lexical semantic resources. Experiments show that our systems can be consistently and significantly improved with rich lexical semantic information, re- gardless of the choice of learning algo- rithms. When evaluated on a bench- mark dataset, the MAP and MRR scores are increased by 8 to 10 points, com- pared to one of our baseline systems using only surface-form matching. Moreover, our best system also outperforms pervious work that makes use of the dependency tree structure by a wide margin. 1 Introduction Open-domain question answering (QA), which fulfills a user’s information need by outputting di- rect answers to natural language queries, is a chal- lenging but important problem (Etzioni, 2011). State-of-the-art QA systems often implement a complicated pipeline architecture, consisting of question analysis, document or passage retrieval, answer selection and verification (Ferrucci, 2012; Moldovan et al., 2003). In this paper, we focus on one of the key subtasks – answer sentence se- lection. Given a question and a set of candidate sentences, the task is to choose the correct sen- tence that contains the exact answer and can suf- ficiently support the answer choice. For instance, although both of the following sentences contain the answer “Jack Lemmon” to the question “Who won the best actor Oscar in 1973?” only the first sentence is correct. A1: Jack Lemmon won the Academy Award for Best Actor for Save the Tiger (1973). A2: Oscar winner Kevin Spacey said that Jack Lemmon is remembered as always making time for other people. One of the benefits of answer sentence selec- tion is that the output can be provided directly to the user. Instead of outputting only the answer, re- turning the whole sentence often adds more value as the user can easily verify the correctness with- out reading a lengthy document. Answer sentence selection can be naturally re- duced to a semantic text matching problem. Con- ceptually, we would like to measure how close the question and sentence can be matched seman- tically. Due to the variety of word choices and inherent ambiguities in natural languages, bag-of- words approaches with simple surface-form word matching tend to produce brittle results with poor prediction accuracy (Bilotti et al., 2007). As a result, researchers put more emphasis on exploit- ing both the syntactic and semantic structure in questions/sentences. Representative examples in- clude methods based on deeper semantic anal- ysis (Shen and Lapata, 2007; Moldovan et al., 2007) and on tree edit-distance (Punyakanok et al., 2004; Heilman and Smith, 2010) and quasi- synchronous grammar (Wang et …

Content truncated to 500 words