Integrating Linguistic Knowledge in Passage Retrieval for Question Answering

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 939–946, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics Integrating linguistic knowledge in passage retrieval for question answering J¨org Tiedemann Alfa Informatica, University of Groningen Oude Kijk in ’t Jatstraat 26 9712 EK Groningen, The Netherlands j.tiedemann@rug.nl Abstract In this paper we investigate the use of lin- guistic knowledge in passage retrieval as part of an open-domain question answer- ing system. We use annotation produced by a deep syntactic dependency parser for Dutch, Alpino, to extract various kinds of linguistic features and syntactic units to be included in a multi-layer index. Sim- ilar annotation is produced for natural lan- guage questions to be answered by the system. From this we extract query terms to be sent to the enriched retrieval index. We use a genetic algorithm to optimize the selection of features and syntactic units to be included in a query. This algo- rithm is also used to optimize further pa- rameters such as keyword weights. The system is trained on questions from the competition on Dutch question answering within the Cross-Language Evaluation Fo- rum (CLEF). We could show an improve- ment of about 15% in mean total recip- rocal rank compared to traditional infor- mation retrieval using plain text keywords (including stemming and stop word re- moval). 1 Introduction Improving information retrieval (IR) through natu- ral language processing (NLP) has been the goal for many researchers. NLP techniques such as lemmatization and compound splitting have been used in several studies (Krovetz, 1993; Hollink et al., 2003). Linguistically motivated syntactic units such as noun phrases (Zhai, 1997), head-modifier pairs (Fagan, 1987; Strzalkowski et al., 1996) and subject- verb-object triples (Katz and Lin, 2003) have also been integrated in information retrieval. However, most of these studies resulted in only little success or even decreasing performance. It has been argued that NLP and especially deep syntactic analysis is still too brittle and ineffective (Katz and Lin, 2003). Integrating NLP in information retrieval seems to be very hard because the task here is to match plain text keywords to natural language documents. In question answering (QA), however, the task is to match natural language questions to relevant an- swers within document collections. For this, we have to analyze the question in order to determine what kind of answer the user is expecting. Tradi- tional information retrieval is used in QA systems to filter out relevant passages from the document col- lection which are then processed to extract possible answers. Hence, the performance of this passage re- trieval component (especially in terms of recall) is crucial for the success of the entire system. NLP tools and linguistic resources are frequently used in QA systems, e.g. (Bernardi et al., 2003; Moldovan et al., 2002), although not very often for passage retrieval (some exceptions are (Strzalkowski et al., 1996; Katz and Lin, 2003; Neumann and Sacaleanu, 2004)). Our goal is to utilize information that can be ex- tracted from …

Content truncated to 500 words