Hierarchical Dirichlet Trees for Information Retrieval

Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the ACL, pages 173–181, Boulder, Colorado, June 2009. c⃝2009 Association for Computational Linguistics Hierarchical Dirichlet Trees for Information Retrieval Gholamreza Haffari School of Computing Sciences Simon Fraser University ghaffar1@cs.sfu.ca Yee Whye Teh Gatsby Computational Neuroscience University College London ywteh@gatsby.ucl.ac.uk Abstract We propose a principled probabilisitc frame- work which uses trees over the vocabulary to capture similarities among terms in an infor- mation retrieval setting. This allows the re- trieval of documents based not just on occur- rences of specific query terms, but also on sim- ilarities between terms (an effect similar to query expansion). Additionally our principled generative model exhibits an effect similar to inverse document frequency. We give encour- aging experimental evidence of the superiority of the hierarchical Dirichlet tree compared to standard baselines. 1 Introduction Information retrieval (IR) is the task of retrieving, given a query, the documents relevant to the user from a large quantity of documents (Salton and McGill, 1983). IR has become very important in recent years, with the proliferation of large quanti- ties of documents on the world wide web. Many IR systems are based on some relevance score function R(j, q) which returns the relevance of document j to query q. Examples of such relevance score functions include term frequency-inverse document frequency (tf-idf) and Okapi BM25 (Robertson et al., 1992). Besides the effect that documents containing more query terms should be more relevant (term fre- quency), the main effect that many relevance scores try to capture is that of inverse document frequency: the importance of a term is inversely related to the number of documents that it appears in, i.e. the popularity of the term. This is because popular terms, e.g. common and stop words, are often un- informative, while rare terms are often very infor- mative. Another important effect is that related or co-occurring terms are often useful in determining the relevance of documents. Because most relevance scores do not capture this effect, IR systems resort to techniques like query expansion which includes syn- onyms and other morphological forms of the origi- nal query terms in order to improve retrieval results; e.g. (Riezler et al., 2007; Metzler and Croft, 2007). In this paper we explore a probabilistic model for IR that simultaneously handles both effects in a prin- cipled manner. It builds upon the work of (Cow- ans, 2004) who proposed a hierarchical Dirichlet document model. In this model, each document is modeled using a multinomial distribution (making the bag-of-words assumption) whose parameters are given Dirichlet priors. The common mean of the Dirichlet priors is itself assumed random and given a Dirichlet hyperprior. (Cowans, 2004) showed that the shared mean parameter induces sharing of infor- mation across documents in the corpus, and leads to an inverse document frequency effect. We generalize the model of (Cowans, 2004) by re- placing the Dirichlet distributions with Dirichlet tree distributions (Minka, 2003), thus we call our model the hierarchical Dirichlet tree. Related terms …

Content truncated to 500 words