RoseMerry: A Baseline Message-level Sentiment Classification System

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 551–555, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics RoseMerry: A Baseline Message-level Sentiment Classification System Huizhi Liang, Richard Fothergill and Timothy Baldwin The University of Melbourne VIC 3010, Melbourne oklianghuizi@gmail.com, r.fothergill@student.unimelb.edu.au, tb@ldwin.net Abstract In this paper, we propose a baseline message- level sentiment classification method, as de- veloped for SemEval-2015 Task 10, Subtask B. This system leverages both hand-crafted features and message-level embedding fea- tures, and uses an SVM classifier for message- level sentiment classification. In pre-training the embedding features, we use one million randomly-selected tweets. We present re- sults over SemEval-2015 Task 10, Subtask B, as well as the Stanford Sentiment Treebank. Our experiments show the effectiveness of our method over both datasets. 1 Introduction The rise of social media such as blogs and micro- blogs (e.g., Twitter) has fueled interest in sentiment analysis (Liu, 2012; Pang and Lee, 2008). One of the most popular settings for carrying out sen- timent analysis is at the sentence level or over in- dividual micro-blog posts, using the simple three- label class set of POSITIVE, NEGATIVE and NEU- TRAL (Liu, 2012; Pang and Lee, 2008; Rosenthal et al., 2014). Sentiment classification has been shown to have utility in various business intelligence ap- plications, including product marketing, identifying new business opportunities, and managing a com- pany’s reputation (Liu, 2012; Pang and Lee, 2008). Learning effective features plays an impor- tant role in building sentiment classification sys- tems (Liu, 2012; Pang and Lee, 2008). For ex- ample, the winning system in the SemEval-2013 message polarity classification task (Nakov et al., 2013) was based on a rich set of hand-tuned features such as word-sentiment association lexicon features, word n-grams, punctuation, and emoticons, which were combined using a simple SVM-based classi- fier (Mohammad et al., 2013). Recently, there has been a surge of interest in representation learning — automatically learning word and document rep- resentations, often in the form of continuous-valued vectors or “embeddings” — using auto-encoders or neural network language models (Mikolov et al., 2013; Le and Mikolov, 2014). Of particular rel- evance to message-level sentiment analysis, Tang et al. (2014) proposed a deep learning approach to learn sentiment-specific word representation fea- tures, and Le and Mikolov (2014) proposed a neu- ral network auto-encoder to learn message-level vec- tors. In this paper, we detail RoseMerry, a (strong) baseline sentiment analysis method that combines hand-crafted features with message-level1 embed- dings generated by doc2vec (Le and Mikolov, 2014), using a linear-kernel SVM. 2 The Proposed Method The proposed method combines a set of hand- crafted features with automatically-generated message-level representation features. The fea- tures are concatenated into a combined feature representation, and fed into a linear-kernel SVM learner using LibSVM (Chang and Lin, 2011). The 1Throughout the paper, we will use “message” as a generic term to refer to both tweets and also sentences in the case of the Stanford Sentiment Treebank. Note that the method could potentially be applied to any …

Content truncated to 500 words