A Self-adaptive Classifier for Efficient Text-stream Processing

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1091–1102, Dublin, Ireland, August 23-29 2014. A Self-adaptive Classifier for Efficient Text-stream Processing Naoki Yoshinaga Institute of Industrial Science, the University of Tokyo, Meguro-ku, Tokyo 153-8505, Japan ynaga@tkl.iis.u-tokyo.ac.jp Masaru Kitsuregawa Institute of Industrial Science, the University of Tokyo, Meguro-ku, Tokyo 153-8505, Japan and National Institute of Informatics, Chiyoda-ku, Tokyo 101-8430, Japan kitsure@tkl.iis.u-tokyo.ac.jp Abstract A self-adaptive classifier for efficient text-stream processing is proposed. The proposed classifier adaptively speeds up its classification while processing a given text stream for various NLP tasks. The key idea behind the classifier is to reuse results for past classification problems to solve forthcoming classification problems. A set of classification problems commonly seen in a text stream is stored to reuse the classification results, while the set size is controlled by removing the least-frequently-used or least-recently-used classification problems. Experimental results with Twitter streams confirmed that the proposed classifier applied to a state-of-the-art base-phrase chunker and dependency parser speeds up its classification by factors of 3.2 and 5.7, respectively. 1 Introduction The rapid growth in popularity of microblogs (e.g., Twitter) is enabling more and more people to in- stantly publish their experiences or thoughts any time they want from mobile devices. Since information in text posted by hundreds of millions of those people covers every space and time in the real world, analyzing such a text stream tells us what is going on in the real world and is therefore beneficial for re- ducing damage caused by natural disasters (Sakaki et al., 2010; Neubig et al., 2011a; Varga et al., 2013), monitoring political sentiment (Tumasjan et al., 2010) and disease epidemics (Aramaki et al., 2011), and predicting stock market (Gilbert and Karahalios, 2010) and criminal incident (Wang et al., 2012). Text-stream processing, however, faces a new challenge; namely, the quality (content) and quantity (volume of flow) changes dramatically, reflecting a change in the real world. Current studies on pro- cessing microblogs have focused mainly on the difference between the quality of microblogs (or spoken languages) and news articles (or written languages) (Gimpel et al., 2011; Foster et al., 2011; Ritter et al., 2011; Han and Baldwin, 2011), and they have not addressed the issue of so-called “bursts” that increase the volume of text. Although it is desirable to use NLP analyzers with the highest possible accuracy for processing a text stream, high accuracy is generally attained by costly structured classification or classi- fication with rich features, typically conjunctive features (Liang et al., 2008). It is therefore inevitable to trade accuracy for speed by using only a small fraction of features to assure real-time processing. In this study, the aforementioned text-quantity issue concerning processing a text stream is addressed, and a self-adaptive algorithm that speeds up an NLP classifier trained with many conjunctive features (or with a polynomial kernel) for a given text stream is proposed and validated. Since globally-observable events such as natural disasters or sports events incur a rapid growth in the number …

Content truncated to 500 words