Complementing WordNet with Roget’s and Corpus-based Thesauri for Information Retrieval

Proceedings of EACL '99 Complementing WordNet with Roget's and Corpus-based Thesauri for Information Retrieval Rila Mandala, Takenobu Tokunaga and Hozumi Tanaka Abstract This paper proposes a method to over- come the drawbacks of WordNet when applied to information retrieval by com- plementing it with Roget's thesaurus and corpus-derived thesauri. Words and rela- tions which are not included in WordNet can be found in the corpus-derived the- sauri. Effects of polysemy can be min- imized with weighting method consider- ing all query terms and all of the the- sauri. Experimental results show that our method enhances information re- trieval performance significantly. Department of Computer Science Tokyo Institute of Technology 2-12-1 Oookayama Meguro-Ku Tokyo 152-8522 Japan {rila,take,tanaka}@cs.titech.ac.jp expansion (Voorhees, 1994; Smeaton and Berrut, 1995), computing lexical cohesion (Stairmand, 1997), word sense disambiguation (Voorhees, 1993), and so on, but the results have not been very successful. Previously, we conducted query expansion ex- periments using WordNet (Mandala et al., to ap- pear 1999) and found limitations, which can be summarized as follows : 1 Introduction Information retrieval (IR) systems can be viewed basically as a form of comparison between doc- uments and queries. In traditional IR methods, this comparison is done based on the use of com- mon index terms in the document and the query (Salton and McGill, 1983). The drawback of such methods is that if semantically relevant docu- ments do not contain the same terms as the query, then they will be judged irrelevant by the IR sys- tem. This occurs because the vocabulary that the user uses is often not the same as the one used in documents (Blair and Maron, 1985). To avoid the above problem, several researchers have suggested the addition of terms which have similar or related meaning to the query, increasing the chances of matching words in relevant docu- ments. This method is called query expansion. A thesaurus contains information pertaining to paradigmatic semantic relations such as term syn- onymy, hypernymy, and hyponymy (Aitchison and Gilchrist, 1987). It is thus natural to use a the- saurus as a source for query expansion. Many researchers have used WordNet (Miller, 1990) in information retrieval as a tool for query • Interrelated words may have different parts of speech. • Most domain-specific relationships between words are not found in WordNet. • Some kinds of words are not included in WordNet, such as proper names. To overcome all the above problems, we pro- pose a method to enrich WordNet with Roget's Thesaurus and corpus-based thesauri. The idea underlying this method is that the automatically constructed thesauri can counter all the above drawbacks of WordNet. For example, as we stated earlier, proper names and their interrelations are not found in WordNet, but if proper names bear some strong relationship with other terms, they often cooccur in documents, as can be modelled by a corpus-based thesaurus. Polysemous words degrade the precision of in- formation retrieval since all senses of the original query term are considered for expansion. To over- come the problem …

Content truncated to 500 words