Language Modeling for Document Selection in Question Answering

Proceedings of Recent Advances in Natural Language Processing, pages 716–720, Hissar, Bulgaria, 12-14 September 2011. Language Modeling for Document Selection in Question Answering Nicolas Foucault, Gilles Adda, Sophie Rosset LIMSI - CNRS firstname.lastname@limsi.fr Abstract Usually, in the Question Answering do- main, for a question in natural language, precise answers to the question are ex- tracted from documents according only to the context of the question. In this work, we complemented this approach by adding a filtering process on top of the docu- ment retrieval. This way, the system re- evaluates the documents it has originally selected during the information retrieval step before the answer extraction and scor- ing. Such re-evaluation aims at filtering out documents considered unusable for the search. Based on statistical language mod- eling, the filtering process firstly deter- mines the intrinsic relevancy of a docu- ment and then decides whether this doc- ument is a priori relevant for finding an- swers. Evaluation on factoid questions and a collection of 500k web documents has shown our approach properly supports the Question Answering task. 1 Introduction Question-Answering (QA) systems can be seen as an extension of the Information Retrieval (IR) en- gines. In IR systems a user is able to search for information using a set of keywords. The search result is a set of documents or links to documents the user needs to peruse to find the precise infor- mation he asked for. In contrast, the QA task con- sists of providing short, relevant answers to nat- ural language questions which can be textual or spoken. For instance, looking for the main ac- tors playing in the ”Titanic” movie directed by James Cameron, a possible question to a QA sys- tem would be: Who did play the main roles in the Titanic movie directed by James Cameron?. In re- turn, the system might reply: Leonardo DiCaprio and Kate Winslet. Question-Answering systems usually follow a standard strategy. They start by preprocessing the documents before their indexation. The indexation for subsequent retrieval is done by a classical (e.g. Lucene1) or specific search en- gine (Rosset et al., 2008) developped on purpose to best fit the system needs. Following these steps which predates any re- trieval, the work turns towards the questions. The question analysis aims at providing information from the question that has to be found in the doc- uments. The second part of the analysis aims to predict what type of answers the question ex- pects (Pardino et al., 2008), usually a named en- tity category (such as person, location, etc.) and also to predict what the question class is, so as to constrain the system to search for specific answer types. The results of these analysis are given to the search engine which retrieves whole documents or snippets, based on the indexation, in order the system finally rank candidates answers it extracted from them. In this paper, we describe a method which first determines the intrinsic relevancy of a document using a language model and then decides whether …

Content truncated to 500 words