SwatCS: Combining simple classifiers with estimated accuracy

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 425–429, Atlanta, Georgia, June 14-15, 2013. c⃝2013 Association for Computational Linguistics SwatCS: Combining simple classifiers with estimated accuracy Sam Clark and Richard Wicentowski Department of Computer Science Swarthmore College Swarthmore, PA 19081 USA sclark2@sccs.swarthmore.edu and richardw@cs.swarthmore.edu Abstract This paper is an overview of the SwatCS system submitted to SemEval-2013 Task 2A: Contextual Polarity Disambiguation. The sen- timent of individual phrases within a tweet are labeled using a combination of classifiers trained on a range of lexical features. The classifiers are combined by estimating the ac- curacy of the classifiers on each tweet. Perfor- mance is measured when using only the pro- vided training data, and separately when in- cluding external data. 1 Introduction Spurred on by the wide-spread use of the social net- works to communicate with friends, fans and cus- tomers around the globe, Twitter has been adopted by celebrities, athletes, politicians, and major com- panies as a platform that mitigates the interaction be- tween individuals. Analysis of this Twitter data can provide insights into how users express themselves. For example, many new forms of expression and language fea- tures have emerged on Twitter, including expres- sions containing mentions, hashtags, emoticons, and abbreviations. This research leverages the lexical features in tweets to predict whether a phrase within a tweet conveys a positive or negative sentiment. 2 Related Work A common goal of past research has been to discover and extract features from tweets that accurately in- dicate sentiment (Liu, 2010). The importance of feature selection and machine learning in sentiment analysis has been explored prior to the rise of so- cial networks. For example, Pang and Lee (2004) apply machine learning techniques to extracted fea- tures from movie reviews. More recent feature-based systems include a lexicon-based approach (Taboada et al., 2011), and a more focused study on the importance of both ad- verbs and adjectives in determining sentiment (Be- namara et al., 2007). Other examples include us- ing looser descriptions of sentiment rather than rigid positive/negative labelings (Whitelaw et al., 2005) and investigating how connections between users can be used to predict sentiment (Tan et al., 2011). This task differs from past work in sentiment anal- ysis of tweets because we aim to build a model capa- ble of predicting the sentiment of sub-phrases within the tweet rather than considering the entire tweet. Specifically, “given a message containing a marked instance of a word or a phrase, determine whether that instance is positive, negative or neutral in that context” (Wilson et al., 2013). Research on context- oriented polarity predates the emergence of social networks: (Nasukawa and Yi, 2003) predict senti- ment of subsections in a larger document. N-gram features, part of speech features and “micro-blogging features” have been used as accu- rate indicators of polarity (Kouloumpis et al., 2011). The “micro-blogging features” are of particular in- terest as they provide insight into how users have adapted Twitter tokens to …

Content truncated to 500 words