Information Extraction for Question Answering: Improving Recall Through Syntactic Patterns

Information Extraction for Question Answering: Improving Recall Through Syntactic Patterns Valentin Jijkoun and Maarten de Rijke Informatics Institute University of Amsterdam {jijkoun,mdr}@science.uva.nl Jori Mur Information Science University of Groningen mur@let.rug.nl Abstract We investigate the impact of the precision/recall trade-off of information extraction on the per- formance of an offline corpus-based question answering (QA) system. One of our findings is that, because of the robust final answer selection mechanism of the QA system, recall is more im- portant. We show that the recall of the extrac- tion component can be improved using syntac- tic parsing instead of more common surface text patterns, substantially increasing the number of factoid questions answered by the QA system. 1 Introduction Current retrieval systems allow us to locate docu- ments that might contain the pertinent information, but most of them leave it to the user to extract the useful information from a ranked list of documents. Hence, the (often unwilling) user is left with a rel- atively large amount of text to consume. There is a need for tools that reduce the amount of text one might have to read to obtain the desired informa- tion. Corpus-based question answering is designed to take a step closer to information retrieval rather than document retrieval. The question answering (QA) task is to find, in a large collection of data, an answer to a question posed in natural language. One particular QA strategy that has proved suc- cessful on large collections uses surface patterns de- rived from the question to identify answers. For ex- ample, for questions like When was Gandhi born?, typical phrases containing the answer are Gandhi was born in 1869 and Gandhi (1869–1948). These examples suggest that text patterns such as “name was born in birth date” and “name (birth year–death year)” formulated as regular ex- pressions, can be used to select the answer phrase. Similarly, such lexical or lexico-syntactic pat- terns can be used to extract specific information on semantic relations from a corpus offline, before ac- tual questions are known, and store it in a repository for quick and easy access. This strategy allows one to handle some frequent question types: Who is... , Where is. . . , What is the capital of. . . etc. (Fleis- chman et al., 2003; Jijkoun et al., 2003). A great deal of work has addressed the problem of extracting semantic relations from unstructured text. Building on this, much recent work in QA has focused on systems that extract answers from large bodies of text using simple lexico-syntactic patterns. These studies indicate two distinct prob- lems associated with using patterns to extract se- mantic information from text. First, the patterns yield only a small subset of the information that may be present in a text (the recall problem). Second, a fraction of the information that the patterns yield is unreliable (the precision problem). The precision of the extracted information can be improved signif- icantly by using machine learning methods to filter out noise (Fleischman et al., 2003). …

Content truncated to 500 words