Offline Strategies for Online Question Answering: Answering Questions Before They Are Asked

Offline Strategies for Online Question Answering: Answering Questions Before They Are Asked Michael Fleischman, Eduard Hovy, Abdessamad Echihabi USC Information Sciences Institute 4676 Admiralty Way Marina del Rey, CA 90292-6695 {fleisch, hovy, echihabi} @ISI.edu Abstract Recent work in Question Answering has focused on web-based systems that extract answers using simple lexico- syntactic patterns. We present an alternative strategy in which patterns are used to extract highly precise relational information offline, creating a data repository that is used to efficiently answer questions. We evaluate our strategy on a challenging subset of questions, i.e. “Who is …” questions, against a state of the art web-based Question Answering system. Results indicate that the extracted relations answer 25% more questions correctly and do so three orders of magnitude faster than the state of the art system. 1 Introduction Many of the recent advances in Question Answering have followed from the insight that systems can benefit by exploiting the redundancy of information in large corpora. Brill et al. (2001) describe using the vast amount of data available on the World Wide Web to achieve impressive performance with relatively simple techniques. While the Web is a powerful resource, its usefulness in Question Answering is not without limits. The Web, while nearly infinite in content, is not a complete repository of useful information. Most newspaper texts, for example, do not remain accessible on the Web for more than a few weeks. Further, while Information Retrieval techniques are relatively successful at managing the vast quantity of text available on the Web, the exactness required of Question Answering systems makes them too slow and impractical for ordinary users. In order to combat these inadequacies, we propose a strategy in which information is extracted automatically from electronic texts offline, and stored for quick and easy access. We borrow techniques from Text Mining in order to extract semantic relations (e.g., concept-instance relations) between lexical items. We enhance these techniques by increasing the yield and precision of the relations that we extract. Our strategy is to collect a large sample of newspaper text (15GB) and use multiple part of speech patterns to extract the semantic relations. We then filter out the noise from these extracted relations using a machine-learned classifier. This process generates a high precision repository of information that can be accessed quickly and easily. We test the feasibility of this strategy on one semantic relation and a challenging subset of questions, i.e., “Who is …” questions, in which either a concept is presented and an instance is requested (e.g., “Who is the mayor of Boston?”), or an instance is presented and a concept is requested (e.g., “Who is Jennifer Capriati?”). By choosing this subset of questions we are able to focus only on answers given by concept-instance relationships. While this paper examines only this type of relation, the techniques we propose are easily extensible to other question types. Evaluations are conducted using a set of “Who is …” questions collected over the period of a few months from the commercial …

Content truncated to 500 words