Bayesian Word Sense Induction

Proceedings of the 12th Conference of the European Chapter of the ACL, pages 103–111, Athens, Greece, 30 March – 3 April 2009. c⃝2009 Association for Computational Linguistics Bayesian Word Sense Induction Samuel Brody Dept. of Biomedical Informatics Columbia University samuel.brody@dbmi.columbia.edu Mirella Lapata School of Informatics University of Edinburgh mlap@inf.ed.ac.uk Abstract Sense induction seeks to automatically identify word senses directly from a cor- pus. A key assumption underlying pre- vious work is that the context surround- ing an ambiguous word is indicative of its meaning. Sense induction is thus typ- ically viewed as an unsupervised cluster- ing problem where the aim is to partition a word’s contexts into different classes, each representing a word sense. Our work places sense induction in a Bayesian con- text by modeling the contexts of the am- biguous word as samples from a multi- nomial distribution over senses which are in turn characterized as distributions over words. The Bayesian framework pro- vides a principled way to incorporate a wide range of features beyond lexical co- occurrences and to systematically assess their utility on the sense induction task. The proposed approach yields improve- ments over state-of-the-art systems on a benchmark dataset. 1 Introduction Sense induction is the task of discovering automat- ically all possible senses of an ambiguous word. It is related to, but distinct from, word sense disam- biguation (WSD) where the senses are assumed to be known and the aim is to identify the intended meaning of the ambiguous word in context. Although the bulk of previous work has been devoted to the disambiguation problem1, there are good reasons to believe that sense induction may be able to overcome some of the issues associ- ated with WSD. Since most disambiguation meth- ods assign senses according to, and with the aid 1Approaches to WSD are too numerous to list; We refer the interested reader to Agirre et al. (2007) for an overview of the state of the art. of, dictionaries or other lexical resources, it is dif- ficult to adapt them to new domains or to lan- guages where such resources are scarce. A re- lated problem concerns the granularity of the sense distinctions which is fixed, and may not be en- tirely suitable for different applications. In con- trast, when sense distinctions are inferred directly from the data, they are more likely to represent the task and domain at hand. There is little risk that an important sense will be left out, or that ir- relevant senses will influence the results. Further- more, recent work in machine translation (Vickrey et al., 2005) and information retrieval (V´eronis, 2004) indicates that induced senses can lead to im- proved performance in areas where methods based on a fixed sense inventory have previously failed (Carpuat and Wu, 2005; Voorhees, 1993). Sense induction is typically treated as an un- supervised clustering problem. The input to the clustering algorithm are instances of the ambigu- ous word with their accompanying contexts (rep- resented by co-occurrence vectors) and the output is a grouping …

Content truncated to 500 words