Word Sense Induction for Novel Sense Detection

Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 591–601, Avignon, France, April 23 - 27 2012. c⃝2012 Association for Computational Linguistics Word Sense Induction for Novel Sense Detection Jey Han Lau,♠♡ Paul Cook,♡ Diana McCarthy, ♣ David Newman,♢ and Timothy Baldwin♠♡ ♠ NICTA Victoria Research Laboratory ♡ Dept of Computer Science and Software Engineering, University of Melbourne ♢ Dept of Computer Science, University of California Irvine ♣ Lexical Computing jhlau@csse.unimelb.edu.au, paulcook@unimelb.edu.au, diana@dianamccarthy.co.uk, newman@uci.edu, tb@ldwin.net Abstract We apply topic modelling to automatically induce word senses of a target word, and demonstrate that our word sense induction method can be used to automatically de- tect words with emergent novel senses, as well as token occurrences of those senses. We start by exploring the utility of stan- dard topic models for word sense induction (WSI), with a pre-determined number of topics (=senses). We next demonstrate that a non-parametric formulation that learns an appropriate number of senses per word ac- tually performs better at the WSI task. We go on to establish state-of-the-art results over two WSI datasets, and apply the pro- posed model to a novel sense detection task. 1 Introduction Word sense induction (WSI) is the task of auto- matically inducing the different senses of a given word, generally in the form of an unsupervised learning task with senses represented as clusters of token instances. It contrasts with word sense disambiguation (WSD), where a fixed sense in- ventory is assumed to exist, and token instances of a given word are disambiguated relative to the sense inventory. While WSI is intuitively appeal- ing as a task, there have been no real examples of WSI being successfully deployed in end-user ap- plications, other than work by Schutze (1998) and Navigli and Crisafulli (2010) in an information re- trieval context. A key contribution of this paper is the successful application of WSI to the lexico- graphical task of novel sense detection, i.e. identi- fying words which have taken on new senses over time. One of the key challenges in WSI is learning the appropriate sense granularity for a given word, i.e. the number of senses that best captures the token occurrences of that word. Building on the work of Brody and Lapata (2009) and others, we approach WSI via topic modelling — using La- tent Dirichlet Allocation (LDA: Blei et al. (2003)) and derivative approaches — and use the topic model to determine the appropriate sense gran- ularity. Topic modelling is an unsupervised ap- proach to jointly learn topics — in the form of multinomial probability distributions over words — and per-document topic assignments — in the form of multinomial probability distributions over topics. LDA is appealing for WSI as it both as- signs senses to words (in the form of topic alloca- tion), and outputs a representation of each sense as a weighted list of words. LDA offers a solu- tion to the question of sense granularity determi- nation via non-parametric formulations, such as a Hierarchical …

Content truncated to 500 words