PU-BCD: Exponential Family Models for the Coarse- and Fine-Grained All-Words Tasks

Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages 272–276, Prague, June 2007. c⃝2007 Association for Computational Linguistics PU-BCD: Exponential Family Models for the Coarse- and Fine-Grained All-Words Tasks Jonathan Chang Princeton University Department of Electrical Engineering jcone@princeton.edu Miroslav Dud´ık, David M. Blei Princeton University Department of Computer Science {mdudik,blei}@cs.princeton.edu Abstract This paper describes an exponential family model of word sense which captures both occurrences and co-occurrences of words and senses in a joint probability distribution. This statistical framework lends itself to the task of word sense disambiguation. We eval- uate the performance of the model in its par- ticipation on the SemEval-2007 coarse- and fine-grained all-words tasks under a variety of parameters. 1 Introduction This paper describes an exponential family model suited to performing word sense disambiguation. Exponential family models are a mainstay of mod- ern statistical modeling (Brown, 1986) and they are widely and successfully used for example in text classification (Berger et al., 1996). In statistical machine learning research, a general methodology and many algorithms were developed for undirected graphical model representation of exponential fam- ilies (Jordan, 2004), providing a solid basis for effi- cient inference. Our model differs from other probabilistic mod- els used for word sense disambiguation in that it captures not only word-sense co-occurrences but also contextual sense-sense co-occurrences, thereby breaking the na¨ıve Bayes assumption. Although spare in the types of features, the model is extremely expressive. Our model has parameters that control for word-sense interaction and sense-sense similar- ity, allowing us to capture many of the salient fea- tures of word and sense use. After fitting the param- eters of our model from a labeled corpus, the task of word sense disambiguation immediately follows by considering the posterior distribution of senses given words. We used this model to participate in SemEval- 2007 on the coarse- and fine-grained all-words tasks. In both of these tasks, a series of sentences are given with certain words tagged. Each competing system must assign a sense from a sense inventory to the tagged words. In both tasks, performance was gauged by comparing the output of each system to human-tagged senses. In the fine-grained task, precision and recall were simply and directly com- puted against the golden annotations. However, in the coarse-grained task, the sense inventory was first clustered semi-automatically with each cluster rep- resenting an equivalence class over senses (Navigli, 2006). Precision and recall were computed against equivalence classes. This paper briefly derives the model and then explores its properties for WSD. We show how common algorithms, such as “dominant sense” and “most frequent sense,” can be expressed in the ex- ponential family framework. We then proceed to present an evaluation of the developed techniques on the SemEval-2007 tasks in which we participated. 2 The model We describe an exponential family model for word sense disambiguation. We posit a joint distribution over words w and senses s. 2.1 Notation We define a document d to be a sequence of words from some lexicon W; for the …

Content truncated to 500 words