Toward Semantics-Based Answer Pinpointing

Toward Semantics-Based Answer Pinpointing Eduard Hovy, Laurie Gerber, Ulf Hermjakob, Chin-Yew Lin, Deepak Ravichandran Information Sciences Institute University of Southern California 4676 Admiralty Way Marina del Rey, CA 90292-6695 USA tel: +1-310-448-8731 {hovy,gerber,ulf,cyl,ravichan}@isi.edu ABSTRACT We describe the treatment of questions (Question-Answer Typology, question parsing, and results) in the Weblcopedia question answering system. 1. INTRODUCTION Several research projects have recently investigated the problem of automatically answering simple questions that have brief phrasal answers (‘factoids’), by identifying and extracting the answer from a large collection of text. The systems built in these projects exhibit a fairly standard structure: they create a query from the user’s question, perform IR with the query to locate (segments of) documents likely to contain an answer, and then pinpoint the most likely answer passage within the candidate documents. The most common difference lies in the pinpointing. Many projects employ a window-based word scoring method that rewards desirable words in the window. They move the window across the candidate answers texts/segments and return the window at the position giving the highest total score. A word is desirable if it is a content word and it is either contained in the question, or is a variant of a word contained in the question, or if it matches the words of the expected answer. Many variations of this method are possible—of the scores, of the treatment of multi- word phrases and gaps between desirable words, of the range of variations allowed, and of the computation of the expected answer words. Although it works to some degree (giving results of up to 30% in independent evaluations), the window-based method has • several quite serious limitations: it cannot pinpoint answer boundaries precisely (e.g., an exact name or noun phrase), • it relies solely on information at the word level, and hence cannot recognize information of the desired type (such as Person or Location), • it cannot locate and compose parts of answers that are distributed over areas wider than the window. Window-based pinpointing is therefore not satisfactory in the long run, even for factoid QA. In this paper we describe work in our Webclopedia project on semantics-based answer pinpointing. Initially, though, recognizing the simplicity and power of the window-based technique for getting started, we implemented a version of it as a fallback method. We then implemented two more sophisticated methods: syntactic- semantic question analysis and QA pattern matching. This involves classification of QA types to facilitate recognition of desired answer types, a robust syntactic-semantic parser to analyze the question and candidate answers, and a matcher that combines word- and parse-tree-level information to identify answer passages more precisely. We expect that the two methods will really show their power when more complex non- factoid answers are sought. In this paper we describe how well the three methods did relative to each other. Section 2 outlines the Webclopedia system. Sections 3, 4, and 5 describe the semantics-based components: a QA Typology, question and answer parsing, and matching. Finally, we outline current work on automatically …

Content truncated to 500 words