SentiME++ at SemEval-2017 Task 4: Stacking State-of-the-Art Classifiers to Enhance Sentiment Classification

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 648–652, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics SentiME++ at SemEval-2017 Task 4: Stacking State-of-the-Art Classifiers to Enhance Sentiment Classification Enrico Palumbo**,*, Efstratios Sygkounas*, Rapha¨el Troncy* and Giuseppe Rizzo** *EURECOM, Sophia Antipolis, France **ISMB, Turin, Italy Abstract In this paper, we describe the participa- tion of the SentiME++ system to the Se- mEval 2017 Task 4A “Sentiment Analysis in Twitter” that aims to classify whether English tweets are of positive, neutral or negative sentiment. SentiME++ is an en- semble approach to sentiment analysis that leverages stacked generalization to auto- matically combine the predictions of five state-of-the-art sentiment classifiers. Sen- tiME++ achieved officially 61.30% F1- score, ranking 12th out of 38 participants. 1 Introduction The SemEval-2017 Task 4 (Rosenthal et al., 2017) focuses on the classification of tweets into posi- tive, neutral and negative sentiment classes. In 2015, the Webis system (Hagen et al., 2015) showed the effectiveness of ensemble methods for sentiment classification by winning the SemEval- 2015 Task 10 “polarity detection” challenge through the combination of four classifiers that had participated to previous editions of SemEval. In 2016, we have combined the original public re- lease of the Webis system with the Stanford Sen- timent System (Socher et al., 2013) using bag- ging, creating the SentiME system (Sygkounas et al., 2016b,a) which won the ESWC2016 Se- mantic Sentiment Analysis challenge. In bagging, the predictions of the classifiers trained on differ- ent bootstrap samples (bags) are simply averaged to obtained a final prediction. In this paper, we propose SentiME++, an enhanced version of the SentiME system that combines the predictions of the base classifiers through stacked generalization. In Section 2, we detail our approach to stack a meta-learner on top of five state-of-the-art senti- ment classifiers to combine their predictions. In Section 3, we describe the experimental setup of our participation to SemEval and we report the re- sults we obtained in Section 4. Finally, we con- clude the paper in Section 5. 2 Approach 2.1 Preliminaries SentiME++ is based on the predictions of five state-of-the-art sentiment classifiers: NRC-Canada: winner of SemEval 2013, trains a linear kernel SVM classifier on a set of linguis- tic and semantic features to extract sentiment from tweets (Mohammad et al., 2013); GU-MLT-LT: 2nd ranked at SemEval 2013, uses a linear classifier trained by stochastic gradient de- scent with hinge loss and elastic net regularization for their predictions on a set of linguistic and se- mantic features (G¨unther and Furrer, 2013); KLUE: 5th ranked at SemEval 2013, feeds a simple bag-of-words model into popular machine learning classifiers such as Naive Bayes, Linear SVM and Maximum Entropy (Proisl et al., 2013); TeamX: winner of SemEval 2014, uses a variety of pre-processors and features, fed into a super- vised machine learning algorithm which utilizes Logistic Regression (Miura et al., 2014); Stanford Sentiment System: one of the sub- systems of the Stanford NLP Core toolkit1, con- tains the Stanford Tree Parser, a …

Content truncated to 500 words