Learning Word Sense Distributions, Detecting Unattested Senses and Identifying Novel Senses Using Topic Models

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 259–270, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Learning Word Sense Distributions, Detecting Unattested Senses and Identifying Novel Senses Using Topic Models Jey Han Lau,♠ Paul Cook,♥ Diana McCarthy,♦ Spandana Gella,♥ and Timothy Baldwin♥ ♠ Dept of Philosophy, King’s College London ♥ Dept of Computing and Information Systems, The University of Melbourne ♦ University of Cambridge jeyhan.lau@gmail.com, paulcook@unimelb.edu.au, diana@dianamccarthy.co.uk, spandanagella@gmail.com, tb@ldwin.net Abstract Unsupervised word sense disambiguation (WSD) methods are an attractive approach to all-words WSD due to their non-reliance on expensive annotated data. Unsuper- vised estimates of sense frequency have been shown to be very useful for WSD due to the skewed nature of word sense distri- butions. This paper presents a fully unsu- pervised topic modelling-based approach to sense frequency estimation, which is highly portable to different corpora and sense inventories, in being applicable to any part of speech, and not requiring a hi- erarchical sense inventory, parsing or par- allel text. We demonstrate the effective- ness of the method over the tasks of pre- dominant sense learning and sense distri- bution acquisition, and also the novel tasks of detecting senses which aren’t attested in the corpus, and identifying novel senses in the corpus which aren’t captured in the sense inventory. 1 Introduction The automatic determination of word sense infor- mation has been a long-term pursuit of the NLP community (Agirre and Edmonds, 2006; Navigli, 2009). Word sense distributions tend to be Zip- fian, and as such, a simple but surprisingly high- accuracy back-off heuristic for word sense dis- ambiguation (WSD) is to tag each instance of a given word with its predominant sense (McCarthy et al., 2007). Such an approach requires knowl- edge of predominant senses; however, word sense distributions — and predominant senses too — vary from corpus to corpus. Therefore, meth- ods for automatically learning predominant senses and sense distributions for specific corpora are re- quired (Koeling et al., 2005; Lapata and Brew, 2004). In this paper, we propose a method which uses topic models to estimate word sense distributions. This method is in principle applicable to all parts of speech, and moreover does not require a parser, a hierarchical sense representation or parallel text. Topic models have been used for WSD in a num- ber of studies (Boyd-Graber et al., 2007; Li et al., 2010; Lau et al., 2012; Preiss and Stevenson, 2013; Cai et al., 2007; Knopp et al., 2013), but our work extends significantly on this earlier work in focusing on the acquisition of prior word sense distributions (and predominant senses). Because of domain differences and the skewed nature of word sense distributions, it is often the case that some senses in a sense inventory will not be attested in a given corpus. A system ca- pable of automatically finding such senses could reduce ambiguity, particularly in domain adapta- tion settings, while retaining rare but nevertheless viable senses. We further propose a method for ap- plying …

Content truncated to 500 words