On-line Trend Analysis with Topic Models: #twitter Trends Detection Topic Model Online

Proceedings of COLING 2012: Technical Papers, pages 1519–1534, COLING 2012, Mumbai, December 2012. On-line Trend Analysis with Topic Models: #twitter trends detection topic model online Je yHan Lau1,2 Nigel Collier3 Timothy Baldwin1,2 (1) Dept of Computing and Information Systems, The University Melbourne, Australia (2) NICTA Victoria Research Laboratory, Australia (3) National Institute of Informatics, Japan jhlau@csse.unimelb.edu.au, collier@nii.ac.jp, tb@ldwin.net ABSTRACT We present a novel topic modelling-based methodology to track emerging events in microblogs such as Twitter. Our topic model has an in-built update mechanism based on time slices and implements a dynamic vocabulary. We first show that the method is robust in detecting events using a range of datasets with injected novel events, and then demonstrate its application in identifying trending topics in Twitter. KEYWORDS: Topic Model, Twitter, Trend Detection, Topic Evolution, Online Processing. 1519 1 Introduction In recent years, microblogs such as Twitter (http://www.twitter.com/) have emerged as a highly popular form of social media. Microblogs provide a platform for users to post short messages (a.k.a. “tweets”) for followers to read in an on- or off-line fashion. One of the major attractions of microblogs is their contextual immediacy, i.e. they provide “here and now” access to users, including the potential to geotag tweets for the location of the post. Twitter is used in a variety of ways, from posting about personal daily life events to finding jobs to keeping up to date with news events.1 As microblogging services have gained in popularity, a myriad of applications that perform analysis of what is “trending” at a given point in time have emerged, with examples including Trendsmap (http://trendsmap.com/), What the Trend (http://whatthetrend.com/), twinitor (http://www.twinitor.com/) and Twendr (http://twendr.com/). There are a number of reasons for tracking trends. At the end-user level, it provides a way for users to identify popular discussions to follow or participate in. From a social science perspective, it provides insights into how new trends emerge, the half-life of trends, and the types of topics commonly discussed in the Twittersphere, etc. Applications that track trends commonly use a keyword-based approach, and provide output in the form of simple terms, hashtags or term n-grams. While these keywords are indicative of the subject of the trending topic, ultimately they fall short in providing users with a fine-grained insights into the nature of the event. This motivated us to look for an alternative means of analysing and presenting trends, and ultimately led us to look at topic models as a potential solution. Our contributions in this work are as follows. We first describe a topic model that processes documents in an on-line fashion. The model has the important properties that it does not grow over time, and can cope with dynamic changes in vocabulary. We then describe a method to measure shifts in the topic model, in order to track emerging events. We demonstrate the robustness and accuracy of the model using a suite of synthetic datasets based on Twitter and data from the TREC Topic Detection and Tracking (TDT) corpus, and then apply …

Content truncated to 500 words