Unsupervised Word Sense Disambiguation Rivaling Supervised Methods

UNSUPERVISED WORD SENSE DISAMBIGUATION RIVALING SUPERVISED METHODS David Yarowsky Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104, USA yarowsky~unagi, ci s. upenn, edu Abstract This paper presents an unsupervised learn- ing algorithm for sense disambiguation that, when trained on unannotated English text, rivals the performance of supervised techniques that require time-consuming hand annotations. The algorithm is based on two powerful constraints - that words tend to have one sense per discourse and one sense per collocation - exploited in an iterative bootstrapping procedure. Tested accuracy exceeds 96%. 1 Introduction This paper presents an unsupervised algorithm that can accurately disambiguate word senses in a large, completely untagged corpus) The algorithm avoids the need for costly hand-tagged training data by ex- ploiting two powerful properties of human language: 1. One sense per collocation: 2 Nearby words provide strong and consistent clues to the sense of a target word, conditional on relative dis- tance, order and syntactic relationship. 2. One sense per discourse: The sense of a tar- get word is highly consistent within any given document. Moreover, language is highly redundant, so that the sense of a word is effectively overdetermined by (1) and (2) above. The algorithm uses these prop- erties to incrementally identify collocations for tar- get senses of a word, given a few seed collocations 1Note that the problem here is sense disambiguation: assigning each instance of a word to established sense definitions (such as in a dictionary). This differs from sense induction: using distributional similarity to parti- tion word instances into clusters that may have no rela- tion to standard sense partitions. 2Here I use the traditional dictionary definition of collocation - "appearing in the same location; a juxta- position of words". No idiomatic or non-compositional interpretation is implied. for each sense, This procedure is robust and self- correcting, and exhibits many strengths of super- vised approaches, including sensitivity to word-order information lost in earlier unsupervised algorithms. 2 One Sense Per Discourse The observation that words strongly tend to exhibit only one sense in a given discourse or document was stated and quantified in Gale, Church and Yarowsky (1992). Yet to date, the full power of this property has not been exploited for sense disambiguation. The work reported here is the first to take advan- tage of this regularity in conjunction with separate models of local context for each word. Importantly, I do not use one-sense-per-discourse as a hard con- straint; it affects the classification probabilistically and can be overridden when local evidence is strong. In this current work, the one-sense-per-discourse hypothesis was tested on a set of 37,232 examples (hand-tagged over a period of 3 years), the same data studied in the disambiguation experiments. For these words, the table below measures the claim's accuracy (when the word occurs more than once in a discourse, how often it takes on the majority sense for the discourse) and applicability (how often the word does occur more than once in a discourse). The one-sense-per-discourse hypothesis: Word …

Content truncated to 500 words