Webis: An Ensemble for Twitter Sentiment Detection

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 582–589, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics Webis: An Ensemble for Twitter Sentiment Detection Matthias Hagen Martin Potthast Michel Büchner Benno Stein Bauhaus-Universität Weimar <first name>.<last name>@uni-weimar.de Abstract We reproduce four Twitter sentiment classi- fication approaches that participated in pre- vious SemEval editions with diverse feature sets. The reproduced approaches are com- bined in an ensemble, averaging the individ- ual classifiers’ confidence scores for the three classes (positive, neutral, negative) and decid- ing sentiment polarity based on these aver- ages. The experimental evaluation on Sem- Eval data shows our re-implementations to slightly outperform their respective originals. Moreover, not too surprisingly, the ensem- ble of the reproduced approaches serves as a strong baseline in the current edition where it is top-ranked on the 2015 test set. 1 Introduction We reproduce four state-of-the-art approaches to classifying the sentiment expressed in a given tweet, and combine the four approaches to an ensem- ble based on the individual classifiers’ confidence scores. In particular, we focus on subtask B of Sem- Eval 2015’s task 10 “Sentiment Analysis in Twitter,” where the goal is to classify the whole tweet as either positive, neutral, or negative. Since the notebook descriptions accompanying submissions to shared tasks are understandably very terse, it is often a chal- lenge to reproduce the results reported. Therefore, we attempt to reproduce the state-of-the-art Twitter sentiment detection algorithms that have been sub- mitted to the aforementioned task in its previous two editions. Furthermore, we combine the reproduced classifiers in an ensemble. Since the individual ap- proaches employ diverse feature sets, the goal of the ensemble is to combine their individual strengths. The paper at hand is a slight extension of the ap- proach from our ECIR 2015 reproducibility track paper (Hagen et al., 2015) such that also text pas- sages are reused. In our ECIR paper, we showed that three selected approaches participating in the Sem- Eval 2013 Twitter sentiment task 2 could be repro- duced from the papers accompanying the individual approaches. Adding the best participant of the re- spective SemEval 2014 task 9 is shown to form a very strong baseline that was not outperformed by the SemEval 2015 participants on the 2015 test data and that also places in the top-10 in the progress test. In Section 2 we briefly describe some related work while in Section 3 we provide more details on the four individual approaches as well as our ensem- ble scheme. Some concluding remarks and an out- look on future work close the paper in Section 4. An experimental evaluation of our approach and an in-depth comparison to the other participants is not included in this paper since it can be found in the task overview (Rosenthal et al., 2015). 2 Related Work Sentiment detection is a classic problem of text clas- sification. Unlike other text classification tasks, the goal is not to identify topics, entities, or authors of a text but …

Content truncated to 500 words