A Topic Model for Word Sense Disambiguation

Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1024–1033, Prague, June 2007. c⃝2007 Association for Computational Linguistics A Topic Model for Word Sense Disambiguation Jordan Boyd-Graber Computer Science Princeton University Princeton, NJ 08540 jbg@princeton.edu David Blei Computer Science Princeton University Princeton, NJ 08540 blei@cs.princeton.edu Xiaojin Zhu Computer Science University of Wisconsin Madison, WI 53706 jerryzhu@cs.wisc.edu Abstract We develop latent Dirichlet allocation with WORDNET (LDAWN), an unsupervised probabilistic topic model that includes word sense as a hidden variable. We develop a probabilistic posterior inference algorithm for simultaneously disambiguating a corpus and learning the domains in which to con- sider each word. Using the WORDNET hi- erarchy, we embed the construction of Ab- ney and Light (1999) in the topic model and show that automatically learned domains improve WSD accuracy compared to alter- native contexts. 1 Introduction Word sense disambiguation (WSD) is the task of determining the meaning of an ambiguous word in its context. It is an important problem in natural language processing (NLP) because effective WSD can improve systems for tasks such as information retrieval, machine translation, and summarization. In this paper, we develop latent Dirichlet alloca- tion with WORDNET (LDAWN), a generative prob- abilistic topic model for WSD where the sense of the word is a hidden random variable that is inferred from data. There are two central advantages to this approach. First, with LDAWN we automatically learn the con- text in which a word is disambiguated. Rather than disambiguating at the sentence-level or the document-level, our model uses the other words that share the same hidden topic across many documents. Second, LDAWN is a fully-fledged generative model. Generative models are modular and can be easily combined and composed to form more com- plicated models. (As a canonical example, the ubiq- uitous hidden Markov model is a series of mixture models chained together.) Thus, developing a gen- erative model for WSD gives other generative NLP algorithms a natural way to take advantage of the hidden senses of words. In general, topic models are statistical models of text that posit a hidden space of topics in which the corpus is embedded (Blei et al., 2003). Given a corpus, posterior inference in topic models amounts to automatically discovering the underlying themes that permeate the collection. Topic models have re- cently been applied to information retrieval (Wei and Croft, 2006), text classification (Blei et al., 2003), and dialogue segmentation (Purver et al., 2006). While topic models capture the polysemous use of words, they do not carry the explicit notion of sense that is necessary for WSD. LDAWN extends the topic modeling framework to include a hidden meaning in the word generation process. In this case, posterior inference discovers both the topics of the corpus and the meanings assigned to each of its words. After introducing a disambiguation scheme based on probabilistic walks over the WORDNET hierar- chy (Section 2), we embed the WORDNET-WALK in a topic model, where each topic is …

Content truncated to 500 words