Studying the Temporal Dynamics of Word Co-occurrences: An Application to Event Detection

Studying the Temporal Dynamics of Word Co-Occurrences: An Application to Event Detection Daniel Preot¸iuc-Pietro1, Srijith P.K.2, Mark Hepple2, Trevor Cohn3 1Computer and Information Science, University of Pennsylvania, USA 2Department of Computer Science, The University of Sheffield, UK 3Computing and Information Systems The University of Melbourne, Australia danielpr@sas.upenn.edu, {pk.srijith, m.r.hepple}@sheffield.ac.uk, t.cohn@unimelb.edu.au Abstract Streaming media provides a number of unique challenges for computational linguistics. This paper studies the temporal variation in word co-occurrence statistics, with application to event detection. We develop a spectral clustering approach to find groups of mutually informative terms occurring in discrete time frames. Experiments on large datasets of tweets show that these groups identify key real world events as they occur in time, despite no explicit supervision. The performance of our method rivals state-of-the-art methods for event detection on F-score, obtaining higher recall at the expense of precision. Keywords: Topic Detection & Tracking, Information Extraction, Information Retrieval, Text Mining 1. Introduction Algorithms based on word co-occurrences have a long tra- dition in NLP and have been used successfully for applica- tions ranging from sentiment analysis to thesaurus learning, collocation extraction and discovering multiword expres- sions (Turney, 2002; Curran, 2004; Sag et al., 2002; Ev- ert, 2005). However, in most cases, these scores have been computed using underlying static corpora and ignoring any temporal variation. In this paper we study the changing behaviour of word co-occurrences over time using social media data. We hy- pothesise that co-occurrences between words will change over time as a response to real world events. Increased so- cial media usage enables us to extract for analysis large scale streaming data, previously largely unavailable to re- searchers. Data arising from these sources, specifically Twitter, has been shown to reflect real world events in a timely fashion (Sakaki et al., 2010). Using this data we ex- tract and analyse co-occurrence statistics over time. To illustrate the temporal evolution of word co-occurrences, consider the example of the word ‘riot’. Using static cor- pora, e.g. newswire or Wikipedia entries, the highest co- occurring words will be syntactically or semantically re- lated words (e.g. ‘city’, ‘police’, ‘riots’). However, if we study the change of frequently co-occurring words with ‘riot’ over the course of 2011, we notice that in January ‘Egypt’ and ‘Lebanon’ commonly co-occur (due to the riots in both countries and the abundance of news and opinions about these events), while in August 2011, ‘riot’ co-occurs more with words like ‘U.K.’, ‘London’ or ‘Hackney’ where a series of riots took place. Moreover, in small time frames, we observe increases of co-occurrences of other seemingly unrelated words (e.g. ‘Bieber’) because of popular opinions or viral messages (‘If Bieber wins, we riot’). Our application is the problem of discovering and clus- tering words specific of events – newsworthy happen- ings (McCreadie et al., 2013) – using their mutual word co- occurrence scores. For computing these scores we use Nor- malised Pointwise Mutual Information (NPMI) (Bouma, 2009). We group tweets in time windows and treat the co-occurrence score in that time …

Content truncated to 500 words