Subject-Dependent Co-Occurrence and Word Sense Disambiguation

SUBJECT-DEPENDENT CO-OCCURRENCE AND WORD SENSE DISAMBIGUATION Joe A. Guthrie,* Louise Guthrie, Yorick Wilks, and Homa Aidinejad Computing Research LabomtoD, Box 30001 New Mexico State University Las Cruces, NM 88003-0001 ABSTRACT We describe a method for obtaining subject-dependent word sets relative to some (subjecO domain. Using the subject classifications given in the machine-readable ver- sion of Longman's Dictionary of Contemporary English, we established subject-dependent co- occurrence links between words of the defining vocabulary to construct these "neighborhoods". Here, we describe the application of these neigh- borhoods to information retrieval, and present a method of word sense disambiguation based on these co-occurrences, an extension of previous work. INTRODUCTION Word associations have been studied for some time in the fields of psycholinguis- tics (by testing human subjects on words), linguistics (where meaning is often based on how words co-occur with each other), and more recently, by researchers in natural language processing (Church and Hanks, 1990; Hindle and Rooth, 1990; Dagan, 1990; McDonald et al., 1990; Wilks et al., 1990) using statistical measures to identify sets of associated words for use in various natural language processing tasks. One of the tasks where the statistical data on associated words has been used with some success is lexical disambiguation. However, associated word sets gathered * Present address: Mathematics Department, University of Texas at k-:l Paso, El Paso, Tx 79968 from a general corpus may contain words that are associated with many different senses. For example, vocabulary associated with the word "bank" includes "money", "rob", "river" and "sand". In this paper, we describe a method for obtaining subject- dependent associated word sets, or "neigh- borhoods" of a given word, relative to a par- ticular (subject) domain. Using the subject classifications of Longman's Dictionary of Contemporary English (LDOCE), we have established subject-dependent co-occurrence finks between words of the defining vocabu- lary to construct these neighborhoods. We will describe the application of these neigh- borhoods to information reuieval, and present a method of word sense disambigua- tion based on these co-occurrences, an extension of previous work. CO-OCCURRENCE NEIGHBORHOODS Words which occur frequently with a given word may be thought of as forming a "neighborhood" of that word. If we can determine which words (i.e. spelling forms) co-occur frequently with each word sense, we can use these neighborhoods to disambi- guate the word in a given text. Assume that we know of only two of the classic senses of the word bank: 1) A repository for money, and 2) A pile of earth on the edge of a river. We can expect the "money" sense of bank to co-occur frequently with such words 146 as "money", "loan", and "robber", while the "fiver" sense would be more frequently asso- ciated with "river", "bridge", and "earth". In order to disambiguate "bank" in a text, we would produce neighborhoods for each sense, and intersect them with the text, our assumption being that the neighborhood which shared more words with the text would determine the correct sense. Varia- tions of this idea appear in …

Content truncated to 500 words