Exploiting the Use of Prior Probabilities for Passage Retrieval in Question Answering

International Conference RANLP 2009 - Borovets, Bulgaria, pages 99–102 Exploiting the use of Prior Probabilities for Passage Retrieval in Question Answering Surya Ganesh, Vasudeva Varma Language Technologies Research Centre, IIIT-Hyderabad, India suryag@research.iiit.ac.in, vv@iiit.ac.in Abstract Document Retrieval assumes that a document is independent of its relevance, and non-relevance. Previous works showed that the same assump- tion is being considered for passage retrieval in the context of Question Answering. In this pa- per, we relax this assumption and describe a method for estimating the prior of a passage being relevant, and non-relevant to a question. These prior probabilities are used in the pro- cess of ranking passages. We also describe a trivial method for identifying relevant and non- relevant text to a question using the Web and AQUAINT corpus as information sources. An empirical evaluation on TREC 2006 Question Answering test set showed that in the context of Question Answering prior probabilities are nec- essary in ranking the passages. 1 Introduction Passage Retrieval is an intermediate step between doc- ument retrieval and answer extraction in a typical Question Answering (QA) system. It reduces the search space for finding an answer from a massive col- lection of documents to a fixed number of passages (say top 100). Unless the answer is present in one of the retrieved passages, QA systems will not find the answer to a given question. So, passage retrieval is considered as one of the most important components of a QA system. The Probability Ranking Principle [13] states that a retrieval system should rank the documents in decreas- ing order of their probability of relevance to the query. According to the Language Modeling [11] decomposi- tion [8] of this ranking principle, the documents should be ranked using the following equation: log rank(D) = log p(Q|D, R) + log p(D|R) p(D|N) (1) Here the first term p(Q|D, R) measures the likelihood of the query given a document that is relevant and Language Modeling is being used to estimate this value. The second term measures the prior proba- bilities of document being relevant, and non relevant. But, document retrieval assumes that a document is independent of its relevance, and non-relevance. So, documents are just ranked based on Language Model- ing i.e., the probability of a query being generated by a document. Previous works [9] [10] showed that the same approach is being used even for passage retrieval in the context of QA. Previously Jagadeesh et al. [5] used prior proba- bilities in Query-Based Multi-Document Summariza- tion task. They defined an entropy based measure called Information Measure to capture the prior of a sentence. This information measure was computed using external information sources like the Web and Wikipedia. Their experimental results showed that prior probabilities are necessary for ranking sentences in the summarization task. We use a similar approach to exploit the use of prior probabilities for passage re- trieval in QA. In this paper we describe a mutual information measure called KullbackLeibler divergence (KL diver- gence) [3] to compute the prior of …

Content truncated to 500 words