The Role of Lexico-Semantic Feedback in Open-Domain Textual Question-Answering

The Role of Lexico-Semantic Feedback in Open-Domain Textual Question-Answering Sanda Harabagiu, Dan Moldovan Marius Pas¸ca, Rada Mihalcea, Mihai Surdeanu, R˘azvan Bunescu, Roxana Gˆırju, Vasile Rus and Paul Mor˘arescu Department of Computer Science and Engineering Southern Methodist University Dallas, TX 75275-0122 � sanda � @engr.smu.edu Abstract This paper presents an open-domain textual Question-Answering system that uses several feedback loops to en- hance its performance. These feedback loops combine in a new way statistical results with syntactic, semantic or pragmatic information derived from texts and lexical databases. The paper presents the contribution of each feed- back loop to the overall performance of 76% human-assessed precise answers. 1 Introduction Open-domain textual Question-Answering (Q&A), as defined by the TREC competitions1, is the task of identifying in large collections of documents a text snippet where the answer to a natural language question lies. The answer is constrained to be found either in a short (50 bytes) or a long (250 bytes) text span. Frequently, keywords extracted from the natural language question are either within the text span or in its immediate vicinity, forming a text para- graph. Since such paragraphs must be identified throughout voluminous collections, automatic and autonomous Q&A systems incorporate an index of the collection as well as a paragraph retrieval mechanism. Recent results from the TREC evaluations ((Kwok et al., 2000) (Radev et al., 2000) (Allen 1The Text REtrieval Conference (TREC) is a series of workshops organized by the National Institute of Standards and Technology (NIST), designed to advance the state-of- the-art in information retrieval (IR) et al., 2000)) show that Information Retrieval (IR) techniques alone are not sufficient for finding an- swers with high precision. In fact, more and more systems adopt architectures in which the seman- tics of the questions are captured prior to para- graph retrieval (e.g. (Gaizauskas and Humphreys, 2000) (Harabagiu et al., 2000)) and used later in extracting the answer (cf. (Abney et al., 2000)). When processing a natural language question two goals must be achieved. First we need to know what is the expected answer type; in other words, we need to know what we are looking for. Sec- ond, we need to know where to look for the an- swer, e.g. we must identify the question keywords to be used in the paragraph retrieval. The expected answer type is determined based on the question stem, e.g. who, where or how much and eventually one of the question concepts, when the stem is ambiguous (for example what), as described in (Harabagiu et al., 2000) (Radev et al., 2000) (Srihari and Li, 2000). However finding question keywords that retrieve all candidate an- swers cannot be achieved only by deriving some of the words used in the question. Frequently, question reformulations use different words, but imply the same answer. Moreover, many equiv- alent answers are phrased differently. In this pa- per we argue that the answer to complex natural language questions cannot be extracted with sig- nificant precision from large collections of texts unless several lexico-semantic feedback loops …

Content truncated to 500 words