Unsupervised Most Frequent Sense Detection using Word Embeddings

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 1238–1243, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Unsupervised Most Frequent Sense Detection using Word Embeddings Sudha Bhingardive Dhirendra Singh Rudra Murthy V Hanumant Redkar and Pushpak Bhattacharyya Department of Computer Science and Engineering, Indian Institute of Technology Bombay. {sudha,dhirendra,rudra,pb}@cse.iitb.ac.in {hanumantredkar}@gmail.com Abstract An acid test for any new Word Sense Disam- biguation (WSD) algorithm is its performance against the Most Frequent Sense (MFS). The field of WSD has found the MFS baseline very hard to beat. Clearly, if WSD researchers had access to MFS values, their striving to bet- ter this heuristic will push the WSD frontier. However, getting MFS values requires sense annotated corpus in enormous amounts, which is out of bounds for most languages, even if their WordNets are available. In this paper, we propose an unsupervised method for MFS detection from the untagged corpora, which exploits word embeddings. We compare the word embedding of a word with all its sense embeddings and obtain the predominant sense with the highest similarity. We observe signif- icant performance gain for Hindi WSD over the WordNet First Sense (WFS) baseline. As for English, the SemCor baseline is bettered for those words whose frequency is greater than 2. Our approach is language and domain independent. 1 Introduction The MFS baseline is often hard to beat for any WSD system and it is considered as the strongest baseline in WSD (Agirre and Edmonds, 2007). It has been observed that supervised WSD approaches gener- ally outperform the MFS baseline, whereas unsu- pervised WSD approaches fail to beat this baseline. The MFS baseline can be easily created if we have a large amount of sense annotated corpora. The fre- quencies of word senses are obtained from the avail- able sense annotated corpora. Creating such a costly resource for all languages is infeasible, looking at the amount of time and money required. Hence, un- supervised approaches have received widespread at- tention as they do not use any sense annotated cor- pora. In this paper, we propose an unsupervised method for MFS detection. We explore the use of word em- beddings for finding the most frequent sense. We have restricted our approach only to nouns. Our ap- proach can be easily ported to various domains and across languages. The roadmap of the paper is as follows. Section 2 describes our approach - ‘UMFS-WE’. Experiments are given in Section 3. Results and Discussions are given in Section 4. Section 5 mentions the related work. Finally, Section 6 concludes the paper and points to future work. 2 Our Approach: UMFS-WE Word Embeddings have recently gained popular- ity among Natural Language Processing community (Bengio et al., 2003; Collobert et al., 2011). They are based on Distributional Hypothesis which works under the assumption that similar words occur in similar contexts (Harris, 1954). Word Embeddings represent each word with a low-dimensional real valued vector with similar words occurring closer in …

Content truncated to 500 words