Social Media Text Classification under Negative Covariate Shift

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2347–2356, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Social Media Text Classification under Negative Covariate Shift Geli Fei and Bing Liu University of Illinois at Chicago Chicago, IL 60607, USA gfei2@uic.edu, liub@cs.uic.edu Abstract In a typical social media content analysis task, the user is interested in analyzing posts of a particular topic. Identifying such posts is often formulated as a classi- fication problem. However, this problem is challenging. One key issue is covariate shift. That is, the training data is not fully representative of the test data. We ob- served that the covariate shift mainly oc- curs in the negative data because topics discussed in social media are highly di- verse and numerous, but the user-labeled negative training data may cover only a small number of topics. This paper pro- poses a novel technique to solve the problem. The key novelty of the tech- nique is the transformation of document representation from the traditional n- gram feature space to a center-based similarity (CBS) space. In the CBS space, the covariate shift problem is sig- nificantly mitigated, which enables us to build much better classifiers. Experiment results show that the proposed approach markedly improves classification. 1 Introduction Applications using social media data, such as reviews, discussion posts, and (micro) blogs are becoming increasingly popular. We observed from our collaborations with social science and health science researchers that in a typical appli- cation, the researcher first need to obtain a set of posts of a particular topic that he/she wants to study, e.g., a political issue. Keyword search is often used as the first step. However, that is not sufficient due to low precision and low recall. A post containing the keyword “politics” may not be a political post while a post that does not con- tain the keyword may be a political post. Thus, text classification is needed to make more so- phisticated decisions to improve accuracy. For classification, the user first manually la- bels a set of relevant posts (positive data) about the political issue and irrelevant posts (negative data) not about the political issue and then builds a classifier by running a learning algorithm, e.g. SVM or naïve Bayes. However, the resulting classifier may not be satisfactory. There may be many reasons. One key reason we observed is that the labeled negative training data is not fully representative of the negative test data. Let the user-interested topic be P (positive), and the set of all other irrelevant topics discussed in a social media source be T = {T1, T2, …, Tn}, which forms the negative data. n is usually large. However, due to the labor-intensive effort of manual labeling, the user can label only a certain number of training posts. Then the labeled nega- tive training posts may cover only a small num- ber of irrelevant topics S of T (S ⊆ T) as nega- tive. Further, due to the highly dynamic nature …

Content truncated to 500 words