NILC_USP: An Improved Hybrid System for Sentiment Analysis in Twitter Messages

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 428–432, Dublin, Ireland, August 23-24, 2014. NILC USP: An Improved Hybrid System for Sentiment Analysis in Twitter Messages Pedro P. Balage Filho, Lucas Avanc¸o, Thiago A. S. Pardo, Maria G. V. Nunes Interinstitutional Center for Computational Linguistics (NILC) Institute of Mathematical and Computer Sciences, University of S˜ao Paulo S˜ao Carlos - SP, Brazil {balage, taspardo, gracan}@icmc.usp.br avanco@usp.br Abstract This paper describes the NILC USP sys- tem that participated in SemEval-2014 Task 9: Sentiment Analysis in Twitter, a re-run of the SemEval 2013 task under the same name. Our system is an improved version of the system that participated in the 2013 task. This system adopts a hybrid classification process that uses three clas- sification approaches: rule-based, lexicon- based and machine learning. We sug- gest a pipeline architecture that extracts the best characteristics from each classi- fier. In this work, we want to verify how this hybrid approach would improve with better classifiers. The improved system achieved an F-score of 65.39% in the Twit- ter message-level subtask for 2013 dataset (+ 9.08% of improvement) and 63.94% for 2014 dataset. 1 Introduction Twitter is an important platform of social com- munication. The analysis of the Twitter messages (tweets) offers a new possibility to understand so- cial behavior. Understanding the sentiment con- tained in such messages showed to be very impor- tant to understand user behavior and also to as- sist market analysis (Java et al., 2007; Kwak et al., 2010). Sentiment analysis, the area in charge of study- ing how sentiments and opinions are expressed in texts, is usually associated with text classification This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence details: http://creativecommons.org/licenses/by/4.0/ tasks. Sentiment classifiers are commonly cate- gorized in two basic approaches: lexicon-based and machine learning approaches (Taboada et al., 2011). A lexicon-based classifier uses a lexicon to provide the polarity, or semantic orientation, of each word or phrase in the text. A machine learn- ing classifier uses features (usually the vocabulary in the texts) obtained from labeled examples to classify the texts according to their polarity. In this paper, we present a hybrid system for sentiment classification in Twitter messages. Our system combines the lexicon-based and machine learning approaches, as well as uses simple rules to aid in the process. Our system participated in SemEval-2014 Task 9: Sentiment Analysis in Twit- ter (Rosenthal et al., 2014), a re-run for the Se- mEval 2013 task under the same name (Nakov et al., 2013). The task goal was to determine the sen- timent contained in tweets. The task included two sub-tasks: a expression-level classification (Task A) and a message-level classification (Task B). Our system participated only in Task B, where, for a given message, it should classify it as positive, negative, or neutral. The system presented is an improved version of the system submitted for Semeval 2013. Our pre- vious system had …

Content truncated to 500 words