Swiss-Chocolate: Combining Flipout Regularization and Random Forests with Artificially Built Subsystems to Boost Text-Classification for Sentiment

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 608–612, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics Swiss-Chocolate: Combining Flipout Regularization and Random Forests with Artificially Built Subsystems to Boost Text-Classification for Sentiment Fatih Uzdilli Zurich University of Applied Sciences Switzerland uzdi@zhaw.ch Martin Jaggi ETH Zurich Switzerland jaggi@inf.ethz.ch Dominic Egger Zurich University of Applied Sciences Switzerland dominicegger@bluewin.ch Pascal Julmy Zurich University of Applied Sciences Switzerland pascal.julmy@gmail.com Leon Derczynski University of Sheffield UK leon@dcs.shef.ac.uk Mark Cieliebak Zurich University of Applied Sciences Switzerland ciel@zhaw.ch Abstract We describe a classifier for predicting message-level sentiment of English micro- blog messages from Twitter. This paper describes our submission to the SemEval- 2015 competition (Task 10). Our approach is to combine several variants of our previous year’s SVM system into one meta-classifier, which was then trained using a random forest. The main idea is that the meta-classifier allows the combination of the strengths and overcome some of the weaknesses of the artificially-built individual classifiers, and adds additional non-linearity. We were also able to improve the linear classifiers by using a new regularization technique we call flipout. 1 Introduction With the availability of huge amounts of user gener- ated text online, the interest in automatic sentiment analysis of text has greatly increased recently in both academia and industry. The goal is to classify a tweet (on the full mes- sage level) into the three classes positive, negative, and neutral. In this paper, we describe our approach using a modified SVM based classifier on short text as in Twitter messages. Our system has participated in the SemEval-2015 Task 10 competition, “Senti- ment Analysis in Twitter, Subtask–B Message Po- This work is licenced under a Creative Commons Attribu- tion 4.0 International License. Page numbers and proceedings footer are added by the organizers. License details: http: //creativecommons.org/licenses/by/4.0/ larity Classification” (Rosenthal et al., 2015). Previ- ous iterations of the evaluation were run in 2013 and 2014. Our Results in the Competition. Our system was ranked 8th out of 40 participants, with an F1-score of 62.61 on the Twitter-2015 test set. The 2015 win- ning team obtained an average F1-score of 64.84. The detailed rankings of our approach were: 4th rank on the LiveJournal data; 6th on the SMS data (2013); 10th on Twitter-2013; 12th on Twitter-2014; and 25th on Twitter Sarcasm. See (Rosenthal et al., 2015) for full details and all results. Data. In the competition, tweets for training and development were provided as tweet IDs. A frac- tion (10-15%) of the tweets were no longer avail- able on Twitter, which made results of the compe- tition not fully comparable. For testing, in addition to last year’s data (tweets, SMS, LiveJournal), new tweets were provided. An overview of the data that we were able to download is given in Table 1. Our Approach. Our system is based on two main ideas. First, we propose a new regularization tech- nique called flipout, which post-processes a trained classifier model for better generalization perfor- mance. Details of this …

Content truncated to 500 words