Reading Documents for Bayesian Online Change Point Detection

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1610–1619, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Reading Documents for Bayesian Online Change Point Detection Taehoon Kim and Jaesik Choi School of Electrical and Computer Engineering Ulsan National Institute of Science and Technology Ulsan, Korea {carpedm20,jaesik}@unist.ac.kr Abstract Modeling non-stationary time-series data for making predictions is a challenging but important task. One of the key is- sues is to identify long-term changes accu- rately in time-varying data. Bayesian On- line Change Point Detection (BO-CPD) algorithms efficiently detect long-term changes without assuming the Markov property which is vulnerable to local sig- nal noise. We propose a Document based BO-CPD (DBO-CPD) model which automatically detects long-term temporal changes of continuous variables based on a novel dynamic Bayesian analysis which combines a non-parametric regression, the Gaussian Process (GP), with generative models of texts such as news articles and posts on social networks. Since texts often include important clues of signal changes, DBO-CPD enables the accurate predic- tion of long-term changes accurately. We show that our algorithm outperforms exist- ing BO-CPDs in two real-world datasets: stock prices and movie revenues. 1 Introduction Time series data depends on the latent dependence structure which changes over time. Thus, sta- tionary parametric models are not appropriate to represent such dynamic non-stationary processes. Change point analysis (Smith, 1975; Stephens, 1994; Chib, 1998; Barry and Hartigan, 1993) fo- cuses on formal frameworks to determine whether a change has taken place without assuming the Markov property which is vulnerable to local sig- nal noise. When change points are identified, each part of the time series is approximated by specified parametric models under the stationary assump- tions. Such change point detection models have successfully been applied to a variety of data, such as stock markets (Chen and Gupta, 1997; Hsu, 1977; Koop and Potter, 2007), analyzing bees’ be- havior (Xuan and Murphy, 2007), forecasting cli- mates (Chu and Zhao, 2004; Zhao and Chu, 2010), and physics experiments (von Toussaint, 2011). However, offline-based change point analysis suf- fers from slow retrospective inference which pre- vents real-time analysis. Bayesian Online Change Point Detection (BO- CPD) (Adams and MacKay, 2007; Steyvers and Brown, 2005; Osborne, 2010; Gu et al., 2013) overcomes this restriction by exploiting efficient online inference algorithms. BO-CPD algorithms efficiently detect long-term changes by analyzing continuous target values with the Gaussian Pro- cess (GP), a non-parametric regression method. The GP-based CPD model is simple and flexible. However, it is not straightforward to utilize rich external data such as texts in news articles and posts in social networks. In this paper, we propose a novel BO-CPD model that improves the detection of change points in continuous signals by incorporating the rich external information implicitly written in texts on top of the long-term change analysis of the GP. In particular, our model finds causes of sig- nal changes in news articles which are influential sources of markets of interests. Given a set of news articles extracted from the …

Content truncated to 500 words