TUGAS: Exploiting unlabelled data for Twitter sentiment analysis

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 673–677, Dublin, Ireland, August 23-24, 2014. TUGAS: Exploiting Unlabelled Data for Twitter Sentiment Analysis Silvio Amir+, Miguel Almeida∗†, Bruno Martins+, Jo˜ao Filgueiras+, and M´ario J. Silva+ +INESC-ID, Instituto Superior T´ecnico, Universidade de Lisboa, Portugal ∗Priberam Labs, Alameda D. Afonso Henriques, 41, 2o, 1000-123 Lisboa, Portugal †Instituto de Telecomunicac¸˜oes, Instituto Superior T´ecnico, Universidade de Lisboa, Portugal samir@inesc-id.pt, miguel.almeida@priberam.pt, bruno.g.martins@tecnico.ulisboa.pt jfilgueiras@inesc-id.pt, mjs@inesc-id.pt Abstract This paper describes our participation in the message polarity classification task of SemEval 2014. We focused on exploiting unlabeled data to improve accuracy, com- bining features leveraging word represen- tations with other, more common features, based on word tokens or lexicons. We analyse the contribution of the different features, concluding that unlabeled data yields significant improvements. 1 Introduction Research in exploiting social media for mea- suring public opinion, evaluating popularity of products and brands, anticipating stock-market trends, or predicting elections showed promising results (O’Connor et al., 2010; Mitchell et al., 2013). However, this type of content poses a par- ticularly challenging problem for text analysis sys- tems. Typical messages show heavy use of Inter- net slang, emoticons and other abbreviations and discourse conventions. The lexical variation intro- duced by this creative use of language, together with the unconventional spelling and occasional typos, leads to very large vocabularies. On the other hand, messages are very short, and there- fore word feature representations tend to become very sparse, degrading the performance of ma- chine learned classifiers. The growing interest in this problem motivated the creation of a shared task for Twitter Sentiment Analysis in the 2013 edition of SemEval. The Message Polarity Classification task was formal- ized as follows: Given a message, decide whether the message is of positive, negative, or neutral sen- timent. For messages conveying both a positive This work is licensed under a Creative Commons Attribution 4.0 International Licence. Page numbers and proceedings footer are added by the organisers. Licence details: http: //creativecommons.org/licenses/by/4.0/ Positive Neutral Negative Train 2014 3230 4109 1265 Tweets 2013 1572 1640 601 Tweets 2014 982 669 202 SMS 2013 492 1207 394 Tweets Sarcasm 2014 33 13 40 LiveJournal 2014 427 411 304 Table 1: Number of examples per class in each SemEval dataset. The first row represents all train- ing data; the other rows are sets used for testing. and negative sentiment, whichever is the stronger sentiment should be chosen (Nakov et al., 2013). We describe our participation on the 2014 edi- tion of this task, for which a set of manually la- belled messages was created. Complying with the Twitter policies for data access, the corpus was distributed as a list of message IDs and each par- ticipant was responsible for downloading the ac- tual tweets. Using the provided script, we col- lected a training set with 8604 tweets. After sub- mission, the 2014 test sets were also made avail- able. Along with the Tweets 2014 test set, evalu- ation was also performed on a set of tweets with sarcasm, …

Content truncated to 500 words