Homonymy and Polysemy in Information Retrieval

Homonymy and Polysemy in Information Retrieval Robert Krovetz NEC Research Institute" 4 Independence Way Princeton, NJ. 08540 krovetz@research.nj.nec.com Abstract This paper discusses research on distin- guishing word meanings in the context of information retrieval systems. We conduc- ted experiments with three sources of evid- ence for making these distinctions: mor- phology, part-of-speech, and phrases. We have focused on the distinction between homonymy and polysemy (unrelated vs. re- lated meanings). Our results support the need to distinguish homonymy and poly- semy. We found: 1) grouping morpholo- gical variants makes a significant improve- ment in retrieval performance, 2) that more than half of all words in a dictionary that differ in part-of-speech are related in mean- ing, and 3) that it is crucial to assign credit to the component words of a phrase. These experiments provide a better understanding of word-based methods, and suggest where natural language processing can provide further improvements in retrieval perform- ance. 1 Introduction Lexical ambiguity is a fundamental problem in nat- ural language processing, but relatively little quant- itative information is available about the extent of the problem, or about the impact that it has on spe- cific applications. We report on our experiments to resolve lexical ambiguity in the context of informa- tion retrieval (IR). Our approach to disambiguation is to treat the information associated with dictionary This paper is based on work that was done at the Center for Intelligent Information Retrieval at the Uni- versity of Massachusetts. It was supported by the Na- tional Science Foundation, Library of Congress, and Department of Commerce raider cooperative agreement number EEC-9209623. I am grateful for their support. senses (morphology. part of speech, and phrases) as multiple sources of evidence. 1 Experiments were de- signed to test each source of evidence independently, and to identify areas of interaction. Our hypothesis is: Hypothesis 1 Resolving lexical ambiguity will lead to an improvement in retrieval performance. There are many issues involved in determining how word senses should be used in information re- trieval. The most basic issue is one of identity -- what is a word sense? In previous work, research- ers have usually made distinctions based on their intuition. This is not satisfactory for two reasons. First, it is difficult to scale up; researchers have gen- erally focused on only two or three words. Second, they have used very coarse grained distinctions (e.g., 'river bank' v. 'commercial bank'). In practice it is often difficult to determine how many senses a word should have, and meanings are often related (Kilgar- rift 91). A related issue is sense granularity. Dictionar- ies often make very fine distinctions between word meanings, and it isn't clear whether these distinc- tions are important in the context of a particular application. For example, the sentence They danced across the lvom is ambiguous with respect to the word dance. It can be paraphrased as They were across the room and they were dancing, or as They crossed the tvom as they danced. The sentence is not. ambiguous …

Content truncated to 500 words