aueb.twitter.sentiment at SemEval-2016 Task 4: A Weighted Ensemble of SVMs for Twitter Sentiment Analysis

Proceedings of SemEval-2016, pages 96–99, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics aueb.twitter.sentiment at SemEval-2016 Task 4: A Weighted Ensemble of SVMs for Twitter Sentiment Analysis Stavros Giorgis, Apostolos Rousas John Pavlopoulos, Prodromos Malakasiotis and Ion Androutsopoulos NLP Group, Department of Informatics Athens University of Economics and Business, Greece Patission 76, GR-104 34 Athens, Greece http://nlp.cs.aueb.gr Abstract This paper describes the system with which we participated in SemEval-2016 Task 4 (Sen- timent Analysis in Twitter) and specifically the Message Polarity Classification subtask. Our system is a weighted ensemble of two sys- tems. The first one is based on a previous sentiment analysis system and uses manually crafted features. The second system of our en- semble uses features based on word embed- dings. Our ensemble was ranked 5th among 34 teams. The source code of our system is publicly available. 1 Introduction This paper describes the system with which we par- ticipated in SemEval-2016 Task 4 (Sentiment Anal- ysis in Twitter) and specifically the Message Polar- ity Classification subtask (Nakov et al., 2016). In this subtask, each tweet is classified as expressing a positive, negative, or no opinion (neutral). Our sys- tem is a weighted ensemble of two systems. The first one is based on a previous sentiment analysis system (Karampatsis et al., 2014) and uses manually crafted features. The second system of our ensemble uses features based on word embeddings (Mikolov et al., 2013; Pennington et al., 2014). Our ensemble was ranked 5th among 34 teams. Section 2 discusses the datasets we used to train and tune our ensemble. Sections 3 and 4 de- scribe our ensemble and its performance respec- tively. Finally, Section 5 concludes and discusses future work. 2 Data For system training and tuning we used 19,305 tweets from the 2016 datasets provided by the organ- isers of SemEval-2016 Task 4, as well as data from SemEval-2013 Task 2. Specifically, the datasets were: – TWtrain16 : train data for SemEval-2016 Task 4, – TWdev16 : development data for SemEval-2016 Task 4, – TWdevtest16 : dev-test data for SemEval-2016 Task 4, – TWtrain13 : train data for SemEval-2013 Task 2, – TWdev13 : development data for SemEval-2013 Task 2. The organisers also provided 6,908 tweets from old SemEval data, to allow system evaluation during development. These data could not be used directly for training or tuning and were the following: – TWdevtest13 : dev-test data for SemEval-2013 Task 2, – TWdevtest14 : dev-test data for SemEval-2014 Task 9, – TWsarcasm14 : tweets containing sarcasm, – SMS13 : SMS messages from 2013, – LJ14 : messages from Live Journal. 96 Avg(CSP1, CSP2) SD1 SD2 SP1 SP2 Subjectivity Detection Sentiment Polarity CSD1 CSD2 CSP1 CSP2 Figure 1: Ensemble of two sentiment polarity classifiers, SP1 and SP2, which are influenced by two subjectivity detection classifiers, SD1 and SD2, respectively. 3 System Overview The main objective of SemEval-2016 Task 4 is to detect sentiment polarity, i.e., to identify whether a message (tweet) expresses positive, negative or no sentiment …

Content truncated to 500 words