UT-DB: An Experimental Study on Sentiment Analysis in Twitter

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 384–389, Atlanta, Georgia, June 14-15, 2013. c⃝2013 Association for Computational Linguistics UT-DB: An Experimental Study on Sentiment Analysis in Twitter Zhemin Zhu Djoerd Hiemstra Peter Apers Andreas Wombacher CTIT Database Group, University of Twente Drienerlolaan 5, 7500 AE, Enschede, The Netherlands {z.zhu, d.hiemstra, p.m.g.apers, A.Wombacher}@utwente.nl Abstract This paper describes our system for participat- ing SemEval2013 Task2-B (Kozareva et al., 2013): Sentiment Analysis in Twitter. Given a message, our system classifies whether the message is positive, negative or neutral senti- ment. It uses a co-occurrence rate model. The training data are constrained to the data pro- vided by the task organizers (No other tweet data are used). We consider 9 types of fea- tures and use a subset of them in our submitted system. To see the contribution of each type of features, we do experimental study on features by leaving one type of features out each time. Results suggest that unigrams are the most im- portant features, bigrams and POS tags seem not helpful, and stopwords should be retained to achieve the best results. The overall results of our system are promising regarding the con- strained features and data we use. 1 Introduction The past years have witnessed the emergence and popularity of short messages such as tweets and SMS messages. Comparing with the traditional gen- res such as newswire data, tweets are very short and use informal grammar and expressions. The short- ness and informality make them a new genre and bring new challenges to sentiment analysis (Pang et al., 2002) as well as other NLP applications such named entity recognition (Habib et al., 2013). Recently a wide range of methods and features have been applied to sentimental analysis over tweets. Go et al. (2009) train sentiment classi- fiers using machine learning methods, such as Naive Bayes, Maximum Entropy and SVMs, with different combinations of features such as unigrams, bigrams and Part-of-Speech (POS) tags. Microblogging fea- tures such as hashtags, emoticons, abbreviations, all- caps and character repetitions are also found help- ful (Kouloumpis et al., 2011). Saif et al. (2012) train Naive Bayes models with semantic features. Also the lexicon prior polarities have been proved very useful (Agarwal et al., 2011). Davidov et al. (2010) utilize hashtags and smileys to build a large- scale annotated tweet dataset automatically. This avoids the need for labour intensive manual anno- tation. Due to the fact that tweets are generated con- stantly, sentiment analysis over tweets has some in- teresting applications, such as predicting stock mar- ket movement (Bollen et al., 2011) and predicting election results (Tumasjan et al., 2010; O’Connor et al., 2010). But there are still some unclear parts in the lit- erature. For example, it is unclear whether using POS tags improves the sentiment analysis perfor- mance or not. Conflicting results are reported (Pak and Paroubek, 2010; Go et al., 2009). It is also a little surprising that not removing …

Content truncated to 500 words