A Unified Model for Word Sense Representation and Disambiguation

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1025–1035, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics A Unified Model for Word Sense Representation and Disambiguation Xinxiong Chen, Zhiyuan Liu, Maosong Sun State Key Laboratory of Intelligent Technology and Systems Tsinghua National Laboratory for Information Science and Technology Department of Computer Science and Technology Tsinghua University, Beijing 100084, China cxx.thu@gmail.com, {lzy, sms}@tsinghua.edu.cn Abstract Most word representation methods assume that each word owns a single semantic vec- tor. This is usually problematic because lexical ambiguity is ubiquitous, which is also the problem to be resolved by word sense disambiguation. In this paper, we present a unified model for joint word sense representation and disambiguation, which will assign distinct representation- s for each word sense.1 The basic idea is that both word sense representation (WS- R) and word sense disambiguation (WS- D) will benefit from each other: (1) high- quality WSR will capture rich informa- tion about words and senses, which should be helpful for WSD, and (2) high-quality WSD will provide reliable disambiguat- ed corpora for learning better sense rep- resentations. Experimental results show that, our model improves the performance of contextual word similarity compared to existing WSR methods, outperforms state- of-the-art supervised methods on domain- specific WSD, and achieves competitive performance on coarse-grained all-words WSD. 1 Introduction Word representation aims to build vectors for each word based on its context in a large corpus, usually capturing both semantic and syntactic information of words. These representations can be used as features or inputs, which are widely employed in information retrieval (Manning et al., 2008), doc- ument classification (Sebastiani, 2002) and other NLP tasks. 1Our sense representations can be downloaded at http: //pan.baidu.com/s/1eQcPK8i. Most word representation methods assume each word owns a single vector. However, this is usual- ly problematic due to the homonymy and polyse- my of many words. To remedy the issue, Reisinger and Mooney (2010) proposed a multi-prototype vector space model, where the contexts of each word are first clustered into groups, and then each cluster generates a distinct prototype vector for a word by averaging over all context vectors with- in the cluster. Huang et al. (2012) followed this idea, but introduced continuous distributed vectors based on probabilistic neural language models for word representations. These cluster-based models conduct unsuper- vised word sense induction by clustering word contexts and, thus, suffer from the following is- sues: • It is usually difficult for these cluster-based models to determine the number of cluster- s. Huang et al. (2012) simply cluster word contexts into static K clusters for each word, which is arbitrary and may introduce mis- takes. • These cluster-based models are typically off- line , so they cannot be efficiently adapted to new senses, new words or new data. • It is also troublesome to find the sense that a word prototype corresponds to; thus, these cluster-based models cannot be directly used to perform word sense disambiguation. In reality, many large knowledge …

Content truncated to 500 words