Swiss-Chocolate: Sentiment Detection using Sparse SVMs and Part-Of-Speech n-Grams

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 601–604, Dublin, Ireland, August 23-24, 2014. Swiss-Chocolate: Sentiment Detection using Sparse SVMs and Part-Of-Speech n-Grams Martin Jaggi ETH Zurich Z¨urich, Switzerland jaggi@inf.ethz.ch Fatih Uzdilli and Mark Cieliebak Zurich University of Applied Sciences Winterthur, Switzerland { uzdi, ciel } @zhaw.ch Abstract We describe a classifier to predict the message-level sentiment of English micro- blog messages from Twitter. This pa- per describes the classifier submitted to the SemEval-2014 competition (Task 9B). Our approach was to build up on the sys- tem of the last year’s winning approach by NRC Canada 2013 (Mohammad et al., 2013), with some modifications and addi- tions of features, and additional sentiment lexicons. Furthermore, we used a sparse (ℓ1-regularized) SVM, instead of the more commonly used ℓ2-regularization, result- ing in a very sparse linear classifier. 1 Introduction With the immense growth of user generated text online, the interest in automatic sentiment analy- sis of text has greatly increased recently in both academia and industry. In this paper, we describe our approach for a modified SVM based classifier for short text as in Twitter messages. Our system has participated in the SemEval-2014 Task 9 competition, “Sentiment Analysis in Twitter, Subtask–B Message Polarity Classification” (Rosenthal et al., 2014). The goal is to classify a tweet (on the full message level) into the three classes positive, negative, and neu- tral. An almost identical competition was already run in 2013. Our Results in the Competition. Our approach was ranked on the 8th place out of the 50 partici- pating submissions, with an F1-score of 67.54 on the Twitter-2014 test set. The 2014 winning team obtained an average F1-score of 70.96. This work is licenced under a Creative Commons Attribution 4.0 International License. Page numbers and proceedings footer are added by the organizers. License details: http: //creativecommons.org/licenses/by/4.0/ (The more detailed rankings of our approach were 4th rank on the LiveJournal data, 5th on the SMS data (2013), 18th on Twitter-2013, and 16th on Twitter Sarcasm, see (Rosenthal et al., 2014) for full details and all results). Data. In the competition, the tweets for training and development were only provided as tweet IDs. A fraction (10-15%) of the tweets were no longer available on twitter, which makes the results of the competition not fully comparable. For testing, in addition to last years data (tweets and SMS), new tweets and data from a surprise domain were pro- vided. An overview of the data, which we were able to download, is shown in Table 1. Table 1: Overview of the data we found available for training, development and testing. Dataset Total Positive Negative Neutral Train (Tweets) 8224 3058 1210 3956 Dev (Tweets) 1417 494 286 637 Test: Twitter2014 1853 982 202 669 Test: Twitter2013 3813 1572 601 1640 Test: SMS2013 2093 492 394 1207 Test: Tw2014Sarcasm 86 33 40 13 Test: LiveJournal2014 1142 427 304 411 2 Description of Our Approach Compared to the previous NRC Canada 2013 approach (Mohammad et al., 2013), our …

Content truncated to 500 words