Multi-Dimensional Feature Merger for Question Answering

Proceedings of COLING 2012: Technical Papers, pages 1–16, COLING 2012, Mumbai, December 2012. Multi-dimensional feature merger for Question Answering Apoorv Agarwal1 J. William Murdock2 Jenni f er Chu-Carroll2 Adam Lall y2 Adit ya Kal yanpur2 (1) Department of Computer Science, Columbia University, New York, NY, U.S.A. (2) IBM T.J. Watson Research Center, Yorktown Heights, NY, U.S.A. apoorv@cs.columbia.edu, {murdockj, jencc, alally, adityakal}@us.ibm.com Abstract In this paper, we introduce new features for question-answering systems. These features are inspired by the fact that justification of the correct answer (out of many candidate answers) may be present in multiple passages. Our features attempt to combine evidence from multiple passages retrieved for a candidate answer. We present results on two data-sets: Jeopardy! and Doctor’s Dilemma. In both data-sets, our features are ranked highest in correlation with gold class (in the training data) and significantly improve the performance of our existing QA system, Watson. Keywords: Question Answering, multi-dimensional feature merger, Watson. 1 1 Introduction Most existing factoid question answering systems adopt search strategies and scoring algorithms with the assumption that a short passage exists in the reference corpus which contains sufficient information to answer each question. This assumption largely holds true for short and focused factoid questions such as those found in the TREC QA track (Voorhees and Tice, 2000). Examples of TREC QA questions include “When did Hawaii become a state?’” and “What strait separates North America from Asia?’” However, some more complex factoid questions contain facts encompassing multiple facets of the answer, which often cannot be found together in a short text passage. Consider the following examples, selected from collections of Jeopardy!1 and Doctor’s Dilemma2 questions, respectively: (1) WHO’S WHO IN SPORTS: Born in 1956, this Swedish tennis player won 6 French Opens & 5 straight Wimbledons (A: Björn Borg) (2) CARDIOLOGY: Murmur associated with this condition is harsh, systolic, diamond- shaped, and increases in intensity with Valsalva (A: Hypertrophic cardiomyopathy) In both examples, information presented in the question can reasonably be expected to be in documents that describe the respective answer entities. However, it is quite unlikely that all the information will be present in one or two adjacent sentences in the document. More specifically, in example (1), we find birth year and nationality information in the basic biographic section of documents about Björn Borg, while statistics about his tennis record can generally be found in a section about Borg’s career. Similarly, for example (2), the descriptions of typical murmurs associated with hypertrophic cardiomyopathy (harsh, systolic, and diamond-shaped) may not fall under the same section as the impact of Valsalva maneuver on the murmur (which is a factor used to distinguish hypertrophic cardiomyopathy from aortic stenosis). As a result, a typical passage retrieved from most reference corpus would cover only a portion of the facts given in the question. These multi-faceted factoid questions present a challenge for existing question answering systems which make the aforementioned assumption. Consider the following short passages relevant to the question in example (2): (2.1 a) Hypertrophic cardiomyopathy generates …

Content truncated to 500 words