unimelb: Topic Modelling-based Word Sense Induction

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 307–311, Atlanta, Georgia, June 14-15, 2013. c⃝2013 Association for Computational Linguistics unimelb: Topic Modelling-based Word Sense Induction Jey Han Lau, Paul Cook and Timothy Baldwin Department of Computing and Information Systems The University of Melbourne jhlau@csse.unimelb.edu.au, paulcook@unimelb.edu.au, tb@ldwin.net Abstract This paper describes our system for shared task 13 “Word Sense Induction for Graded and Non-Graded Senses” of SemEval-2013. The task is on word sense induction (WSI), and builds on earlier SemEval WSI tasks in ex- ploring the possibility of multiple senses be- ing compatible to varying degrees with a sin- gle contextual instance: participants are asked to grade senses rather than selecting a sin- gle sense like most word sense disambigua- tion (WSD) settings. The evaluation measures are designed to assess how well a system per- ceives the different senses in a contextual in- stance. We adopt a previously-proposed WSI methodology for the task, which is based on a Hierarchical Dirichlet Process (HDP), a non- parametric topic model. Our system requires no parameter tuning, uses the English ukWaC as an external resource, and achieves encour- aging results over the shared task. 1 Introduction In our previous work (Lau et al., 2012) we devel- oped a word-sense induction (WSI) system based on topic modelling, specifically a Hierarchical Dirich- let Process (Teh et al., 2006). In evaluations over the SemEval-2007 and SemEval-2010 WSI tasks we achieved performance on par with the current state- of-the art. The SemEval-2007 and SemEval-2010 WSI tasks assumed that each usage of a word has a single gold-standard sense. In this paper we apply this WSI method “off-the-shelf”, with no adaptation, to the novel SemEval-2013 task of “Word Sense In- duction for Graded and Non-Graded Senses”. Given that the topic model allocates a multinomial distri- bution over topics to each word usage (“document”, in topic modelling terms), the SemEval-2013 WSI task is an ideal means for evaluating this aspect of the topic model. 2 System Description Our system is based on the WSI methodology pro- posed by Lau et al. (2012), and also applied to SemEval-2013 Task 11 on WSI for web snippet clustering (Lau et al., to appear). The core machin- ery of our system is driven by a Latent Dirichlet Al- location (LDA) topic model (Blei et al., 2003). In LDA, the model learns latent topics for a collection of documents, and associates these latent topics with every document in the collection. A topic is repre- sented by a multinomial distribution of words, and the association of topics with documents is repre- sented by a multinomial distribution of topics, a dis- tribution for each document. The generative process of LDA for drawing word w in document d is as fol- lows: 1. draw latent topic z from document d; 2. draw word w from the chosen latent topic z. The probability of selecting word w given a doc- ument d is thus given by: P(w|d) = …

Content truncated to 500 words