Naive Bayes Word Sense Induction

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1433–1437, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Naive Bayes Word Sense Induction Do Kook Choe Brown University Providence, RI dc65@cs.brown.edu Eugene Charniak Brown University Providence, RI ec@cs.brown.edu Abstract We introduce an extended naive Bayes model for word sense induction (WSI) and apply it to a WSI task. The extended model incorporates the idea the words closer to the target word are more relevant in predicting its sense. The pro- posed model is very simple yet effective when evaluated on SemEval-2010 WSI data. 1 Introduction The task of word sense induction (WSI) is to find clusters of tokens of an ambiguous word in an un- labeled corpus that have the same sense. For in- stance, given a target word “crane,” a good WSI sys- tem should find a cluster of tokens referring to avian cranes and another referring to mechanical cranes. We believe that neighboring words contain enough information that these clusters can be found from plain texts. WSI is related to word sense disambiguation (WSD). In a WSD task, a system learns a sense clas- sifier in a supervised manner from a sense-labeled corpus. The performance of the learned classifier is measured on some unseen data. WSD systems perform better than WSI systems, but building la- beled data can be prohibitively expensive. In addi- tion, WSD systems are not suitable for newly cre- ated words, new senses of existing words, or domain- specific words. On the other hand, WSI systems can learn new senses of words directly from texts because these programs do not rely on a predefined set of senses. In Section 2 we describe relevant previous work. In Section 3 and 4 we introduce the naive Bayes model for WSI and inference schemes for the model. In Sec- tion 5 we evaluate the model on SemEval-2010 data. In Section 6 we conclude. 2 Related Work Yarowsky (1995) introduces a semi-supervised bootstrapping algorithm with two assumptions that rivals supervised algorithms: one-sense-per- collocation and one-sense-per-discourse. But this algorithm cannot easily be scaled up because for any new ambiguous word humans need to pick a few seed words, which initialize the algorithm. In order to automate the semi-supervised system, Eisner and Karakos (2005) propose an unsupervised bootstrapping algorithm. Their system tries many different seeds for bootstrapping and chooses the “best” classifier at the end. Eisner and Karakos’s algorithm is limited in that their system is designed for disambiguating words that have only 2 senses. Bayesian WSI systems have been developed by several authors. Brody and Lapata (2009) apply Latent Dirichlet Allocation (LDA) (Blei et al., 2003) to WSI. They run a topic modeling algorithm on texts with some fixed number of topics that correspond to senses and induce a cluster by finding target words assigned to the same topic. Their system is evaluated on SemEval-2007 noun data (Agirre and Soroa, 2007). Lau et al. (2012) apply a nonparametric model, Hierarchical Dirichlet Pro- cesses …

Content truncated to 500 words