Topic Models for Word Sense Disambiguation and Token-Based Idiom Detection

Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1138–1147, Uppsala, Sweden, 11-16 July 2010. c⃝2010 Association for Computational Linguistics Topic Models for Word Sense Disambiguation and Token-based Idiom Detection Linlin Li, Benjamin Roth, and Caroline Sporleder Saarland University, Postfach 15 11 50 66041 Saarbr¨ucken, Germany {linlin, beroth, csporled}@coli.uni-saarland.de Abstract This paper presents a probabilistic model for sense disambiguation which chooses the best sense based on the conditional probability of sense paraphrases given a context. We use a topic model to decom- pose this conditional probability into two conditional probabilities with latent vari- ables. We propose three different instanti- ations of the model for solving sense dis- ambiguation problems with different de- grees of resource availability. The pro- posed models are tested on three different tasks: coarse-grained word sense disam- biguation, fine-grained word sense disam- biguation, and detection of literal vs. non- literal usages of potentially idiomatic ex- pressions. In all three cases, we outper- form state-of-the-art systems either quan- titatively or statistically significantly. 1 Introduction Word sense disambiguation (WSD) is the task of automatically determining the correct sense for a target word given the context in which it occurs. WSD is an important problem in NLP and an es- sential preprocessing step for many applications, including machine translation, question answering and information extraction. However, WSD is a difficult task, and despite the fact that it has been the focus of much research over the years, state- of-the-art systems are still often not good enough for real-world applications. One major factor that makes WSD difficult is a relative lack of manu- ally annotated corpora, which hampers the perfor- mance of supervised systems. To address this problem, there has been a significant amount of work on unsupervised WSD that does not require manually sense- disambiguated training data (see McCarthy (2009) for an overview). Recently, several researchers have experimented with topic models (Brody and Lapata, 2009; Boyd-Graber et al., 2007; Boyd- Graber and Blei, 2007; Cai et al., 2007) for sense disambiguation and induction. Topic models are generative probabilistic models of text corpora in which each document is modelled as a mixture over (latent) topics, which are in turn represented by a distribution over words. Previous approaches using topic models for sense disambiguation either embed topic features in a supervised model (Cai et al., 2007) or rely heavily on the structure of hierarchical lexicons such as WordNet (Boyd-Graber et al., 2007). In this paper, we propose a novel framework which is fairly resource-poor in that it requires only 1) a large unlabelled corpus from which to estimate the topics distributions, and 2) paraphrases for the possible target senses. The paraphrases can be user-supplied or can be taken from existing re- sources. We approach the sense disambiguation task by choosing the best sense based on the conditional probability of sense paraphrases given a context. We propose three models which are suitable for different situations: Model I requires knowledge of the prior distribution over senses and directly maximizes the conditional …

Content truncated to 500 words