Sense Embedding Learning for Word Sense Induction

Proceedings of the Fifth Joint Conference on Lexical and Computational Semantics (*SEM 2016), pages 85–90, Berlin, Germany, August 11-12, 2016. Sense Embedding Learning for Word Sense Induction Linfeng Song1, Zhiguo Wang2, Haitao Mi2 and Daniel Gildea1 1Department of Computer Science, University of Rochester, Rochester, NY 14627 2IBM T.J. Watson Research Center, Yorktown Heights, NY 10598 Abstract Conventional word sense induction (WSI) methods usually represent each instance with discrete linguistic features or co- occurrence features, and train a model for each polysemous word individually. In this work, we propose to learn sense embeddings for the WSI task. In the training stage, our method induces sev- eral sense centroids (embedding) for each polysemous word. In the testing stage, our method represents each instance as a contextual vector, and induces its sense by finding the nearest sense centroid in the embedding space. The advantages of our method are (1) distributed sense vectors are taken as the knowledge rep- resentations which are trained discrimi- natively, and usually have better perfor- mance than traditional count-based distri- butional models, and (2) a general model for the whole vocabulary is jointly trained to induce sense centroids under the mutli- task learning framework. Evaluated on SemEval-2010 WSI dataset, our method outperforms all participants and most of the recent state-of-the-art methods. We further verify the two advantages by com- paring with carefully designed baselines. 1 Introduction Word sense induction (WSI) is the task of auto- matically finding sense clusters for polysemous words. In contrast, word sense disambiguation (WSD) assumes there exists an already-known sense inventory, and the sense of a word type is disambiguated according to the sense inventory. Therefore, clustering methods are generally ap- plied in WSI tasks, while classification methods are utilized in WSD tasks. WSI has been success- fully applied to many NLP tasks such as machine translation (Xiong and Zhang, 2014), information retrieval (Navigli and Crisafulli, 2010) and novel sense detection (Lau et al., 2012). However, existing methods usually represent each instance with discrete hand-crafted features (Bordag, 2006; Chen et al., 2009; Van de Cruys and Apidianaki, 2011; Purandare and Pedersen, 2004), which are designed manually and require linguistic knowledge. Most previous methods re- quire learning a specific model for each polyse- mous word, which limits their usability for down- stream applications and loses the chance to jointly learn senses for multiple words. There is a great advance in recent distributed semantics, such as word embedding (Mikolov et al., 2013; Pennington et al., 2014) and sense em- bedding (Reisinger and Mooney, 2010; Huang et al., 2012; Jauhar et al., 2015; Rothe and Sch¨utze, 2015; Chen et al., 2014; Tian et al., 2014). Comparing with word embedding, sense embed- ding methods learn distributed representations for senses of a polysemous word, which is similar to the sense centroid of WSI tasks. In this work, we point out that the WSI task and the sense embedding task are highly inter- related, and propose to jointly learn sense cen- troids (embeddings) of all polysemous words for the WSI task. …

Content truncated to 500 words