Unsupervised Word Sense Induction using Distributional Statistics

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1302–1310, Dublin, Ireland, August 23-29 2014. Unsupervised Word Sense Induction using Distributional Statistics Kartik Goyal Carnegie Mellon Uniersity kartikgo@cs.cmu.edu Eduard Hovy Carnegie Mellon University hovy@cmu.edu Abstract Word sense induction is an unsupervised task to find and characterize different senses of polyse- mous words. This work investigates two unsupervised approaches that focus on using distribu- tional word statistics to cluster the contextual information of the target words using two different algorithms involving latent dirichlet allocation and spectral clustering. Using a large corpus for achieving this task, we quantitatively analyze our clusters on the Semeval-2010 dataset and also perform a qualitative analysis of our induced senses. Our results indicate that our methods suc- cessfully characterized the senses of the target words and were also able to find unconventional senses for those words. 1 Introduction Word Sense Induction (WSI) involves automatically determining the number of senses of a given word or a phrase and identifying the features which differentiate those senses. This task, although similar to the Word Sense Disambiguation (WSD) task, is fundamentally different because it does not involve any supervision or explicit human knowledge about senses of words. WSI has potential to be extremely useful in downstream applications because, apart from the savings on annotation costs, it also mitigates several theoretical conflicts associated with supervised WSD tasks, which generally involve deciding on the granularity of senses. Ideally, a WSI algorithm would be able to adapt to different tasks requiring different sense granularities. WSI algorithms can also be used to model the evolution of the senses of a word with time and hence can be much easier to maintain than existing fixed sense inventories like WordNet(Miller, 1995), Ontonotes(Hovy et al., 2006) etc. Automatic sense identification systems also have the potential to generalize well to large amounts of diverse data and hence be useful in various difficult domain independent tasks such as machine translation and information retrieval. Several factors make the problem of word sense induction very challenging. Most importantly, it is not clear what should be the ‘true’ senses of a word. The semantic continuum makes it always possible to break a sense into finer grained subsenses. Thus, the problem is one of finding the optimal granularity for any given task. Even in a semi-supervised setting, it is unknown which sense inventories are most suited as starting points in a sense bootstrapping procedure. Our unsupervised approach relies heavily on the distributional statistics of words which occur in the proximity of the target words. Hence, we first obtain the distributional statistics from a very large cor- pus to facilitate generalization and reliable estimation of different possible senses. Then we use these statistics in a novel manner to obtain a representation for the senses of the target word. In this paper, we discus the performance of induced senses on the Semeval 2010 WSD/WSI(Manandhar et al., 2010) task. 2 Related Work Much of the work on word sense induction has been …

Content truncated to 500 words