SU-FMI: System Description for SemEval-2014 Task 9 on Sentiment Analysis in Twitter

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 590–595, Dublin, Ireland, August 23-24, 2014. SU-FMI: System Description for SemEval-2014 Task 9 on Sentiment Analysis in Twitter Boris Velichkov∗, Borislav Kapukaranov†, Ivan Grozev‡, Jeni Karanesheva§, Todor Mihaylov¶, Yasen Kiprov∥, Georgi Georgiev∗∗, Ivan Koychev††, Preslav Nakov‡‡ Abstract We describe the submission of the team of the Sofia University to SemEval-2014 Task 9 on Sentiment Analysis in Twit- ter. We participated in subtask B, where the participating systems had to predict whether a Twitter message expresses pos- itive, negative, or neutral sentiment. We trained an SVM classifier with a linear kernel using a variety of features. We used publicly available resources only, and thus our results should be easily replicable. Overall, our system is ranked 20th out of 50 submissions (by 44 teams) based on the average of the three 2014 evaluation data scores, with an F1-score of 63.62 on gen- eral tweets, 48.37 on sarcastic tweets, and 68.24 on LiveJournal messages. 1 Introduction We describe the submission of the team of the Sofia University, Faculty of Mathematics and In- formatics (SU-FMI) to SemEval-2014 Task 9 on Sentiment Analysis in Twitter (Rosenthal et al., 2014). ∗Sofia University, bobby.velichkov@gmail.com †Sofia University, b.kapukaranov@gmail.com ‡Sofia University, iigrozev@gmail.com §Sofia University, j.karanesheva@gmail.com ¶Sofia University, tbmihailov@gmail.com ∥Sofia University, yasen.kiprov@gmail.com ∗∗ Ontotext, g.d.georgiev@gmail.com ††Sofia University, koychev@fmi.uni-sofia.bg ‡‡Qatar Computing Research Institute, pnakov@qf.org.qa This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence details: http://creativecommons.org/licenses/by/4.0/ This SemEval challenge had two subtasks: • subtask A (term-level) asks to predict the sen- timent of a phrase inside a tweet; • subtask B (message-level) asks to predict the overall sentiment of a tweet message. In both subtasks, the sentiment can be positive, negative, or neutral. Here are some examples: • positive: Gas by my house hit $3.39!!!! I’m going to Chapel Hill on Sat. :) • neutral: New York Giants: Game-by-Game Predictions for the 2nd Half of the Season http://t.co/yK9VTjcs • negative: Why the hell does Selma have school tomorrow but Parlier clovis & others don’t? • negative (sarcastic): @MetroNorth wall to wall people on the platform at South Nor- walk waiting for the 8:08. Thanks for the Sat. Sched. Great sense Below we first describe our preprocessing, fea- tures and classifier in Section 2. Then, we discuss our experiments, results and analysis in Section 3. Finally, we conclude with possible directions for future work in Section 4. 2 Method Our approach is inspired by the highest scoring team in 2013, NRC Canada (Mohammad et al., 2013). We reused many of their resources.1 Our system consists of two main submodules, (i) feature extraction in the framework of GATE (Cunningham et al., 2011), and (ii) machine learn- ing using SVM with linear kernels as implemented in LIBLINEAR2 (Fan et al., 2008). 1http://www.umiacs.umd.edu/ ˜saif/WebPages/Abstracts/ NRC-SentimentAnalysis.htm 2http://www.csie.ntu.edu.tw/˜cjlin/ liblinear/ 590 2.1 Preprocessing We integrated a pipeline of various resources for tweet analysis that are already available in GATE (Bontcheva et …

Content truncated to 500 words