Splusplus: A Feature-Rich Two-stage Classifier for Sentiment Analysis of Tweets

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 515–519, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics Splusplus: A Feature-Rich Two-stage Classifier for Sentiment Analysis of Tweets Li Dong†∗, Furu Wei‡, Yichun Yin§∗, Ming Zhou‡, and Ke Xu† †Beihang University, Beijing, 100191, China ‡Microsoft Research, Beijing, 100080, China §Peking University, Beijing, 100871, China dl@cse.buaa.edu.cn {fuwei,mingzhou}@microsoft.com yichunyin@pku.edu.cn kexu@nlsde.buaa.edu.cn Abstract This paper describes our sentiment classifica- tion system submitted to SemEval-2015 Task 10. In the message-level polarity classifica- tion subtask, we obtain the highest macro- averaged F1-scores on three out of six test- ing sets. Specifically, we build a two-stage classifier to predict the sentiment labels for tweets, which enables us to design different features for subjective/objective classification and positive/negative classification. In addi- tion to n-grams, lexicons, word clusters, and twitter-specific features, we develop several deep learning methods to automatically ex- tract features for the message-level sentiment classification task. Moreover, we propose a polarity boosting trick which improves the performance of our system. 1 Introduction In the task 10 of SemEval-2015, submitted sys- tems are required to categorize tweets to posi- tive, negative, and neutral classes (Rosenthal et al., 2015). There are six testing sets in SemEval- 2015. Four of them are tweets: Twitter13, Twit- ter14, Twitter14Sarcasm, and Twitter15. The Twit- terSarcasm14 consists of the tweets which express sarcasm. In order to evaluate the performance on out-of-domain data, the other two datasets are Live- Journal14 and SMS13 that are from web blogs and SMS messages respectively. The details of these datasets are described in (Nakov et al., 2013; Rosen- thal et al., 2014; Rosenthal et al., 2015). ∗Contribution during internship at Microsoft Research. We utilize both basic features and deep learn- ing features in our system. Deep learning is used to automatically learn representations, which has achieved some promising results on sentiment anal- ysis (Kim, 2014; Socher et al., 2013; Dong et al., 2014). In order to design more flexible fea- tures, we use a two-stage classification framework which conducts subjective/objective (sub/obj) clas- sification and positive/negative (pos/neg) classifica- tion. In addition, we introduce a polarity boosting trick that can utilize pos/neg training data to improve classifying tweets to sub/obj. With the help of these features and methods, our system achieves the best results on three out of six datasets among 40 teams in SemEval-2015. We describe the basic features and deep learning features used in our system, and compare their contributions. Moreover, we make the word2vec clustering results on Twitter data publicly available for research purpose. 2 System Description 2.1 Overview As shown in Figure 1, our sentiment analysis system is a two-stage sentiment classifier which consists of a subjective/objective (sub/obj) classifier and a pos- itive/negative (pos/neg) classifier. By using this ar- titacture, we can design different feature sets for the two classification steps. Notably, the predicted val- ues of pos/neg classifier is employed to help clas- sify tweets to sub/obj classes. We employ the LIB- LINEAR (Fan et al., 2008) with option “-s 1” as …

Content truncated to 500 words