Automatic Labelling of Topic Models

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 1536–1545, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Automatic Labelling of Topic Models Jey Han Lau,♠♥ Karl Grieser,♥ David Newman,♠♦ and Timothy Baldwin♠♥ ♠ NICTA Victoria Research Laboratory ♥ Dept of Computer Science and Software Engineering, University of Melbourne ♦ Dept of Computer Science, University of California Irvine jhlau@csse.unimelb.edu.au, kgrieser@csse.unimelb.edu.au, newman@uci.edu, tb@ldwin.net Abstract We propose a method for automatically la- belling topics learned via LDA topic models. We generate our label candidate set from the top-ranking topic terms, titles of Wikipedia ar- ticles containing the top-ranking topic terms, and sub-phrases extracted from the Wikipedia article titles. We rank the label candidates us- ing a combination of association measures and lexical features, optionally fed into a super- vised ranking model. Our method is shown to perform strongly over four independent sets of topics, significantly better than a benchmark method. 1 Introduction Topic modelling is an increasingly popular frame- work for simultaneously soft-clustering terms and documents into a fixed number of “topics”, which take the form of a multinomial distribution over terms in the document collection (Blei et al., 2003). It has been demonstrated to be highly ef- fective in a wide range of tasks, including multi- document summarisation (Haghighi and Vander- wende, 2009), word sense discrimination (Brody and Lapata, 2009), sentiment analysis (Titov and McDonald, 2008), information retrieval (Wei and Croft, 2006) and image labelling (Feng and Lapata, 2010). One standard way of interpreting a topic is to use the marginal probabilities p(wi|tj) associated with each term wi in a given topic tj to extract out the 10 terms with highest marginal probability. This results in term lists such as:1 stock market investor fund trading invest- ment firm exchange companies share 1Here and throughout the paper, we will represent a topic tj via its ranking of top-10 topic terms, based on p(wi|tj). which are clearly associated with the domain of stock market trading. The aim of this research is to automatically generate topic labels which explicitly identify the semantics of the topic, i.e. which take us from a list of terms requiring interpretation to a sin- gle label, such as STOCK MARKET TRADING in the above case. The approach proposed in this paper is to first generate a topic label candidate set by: (1) sourc- ing topic label candidates from Wikipedia by query- ing with the top-N topic terms; (2) identifying the top-ranked document titles; and (3) further post- processing the document titles to extract sub-strings. We translate each topic label into features extracted from Wikipedia, lexical association with the topic terms in Wikipedia documents, and also lexical fea- tures for the component terms. This is used as the basis of a support vector regression model, which ranks each topic label candidate. Our contributions in this work are: (1) the genera- tion of a novel evaluation framework and dataset for topic label evaluation; (2) the proposal of a method for both generating and scoring topic …

Content truncated to 500 words