KLUE: Simple and robust methods for polarity classification

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 395–401, Atlanta, Georgia, June 14-15, 2013. c⃝2013 Association for Computational Linguistics KLUE: Simple and robust methods for polarity classification Thomas Proisl and Paul Greiner and Stefan Evert and Besim Kabashi Friedrich-Alexander-Universität Erlangen-Nürnberg Department Germanistik und Komparatistik Professur für Korpuslinguistik Bismarckstr. 6 91054 Erlangen, Germany {thomas.proisl,paul.greiner,stefan.evert,besim.kabashi}@fau.de Abstract This paper describes our approach to the SemEval-2013 task on “Sentiment Analysis in Twitter”. We use simple bag-of-words mod- els, a freely available sentiment dictionary auto- matically extended with distributionally similar terms, as well as lists of emoticons and inter- net slang abbreviations in conjunction with fast and robust machine learning algorithms. The resulting system is resource-lean, making it rel- atively independent of a specific language. De- spite its simplicity, the system achieves compet- itive accuracies of 0.70–0.72 in detecting the sentiment of text messages. We also apply our approach to the task of detecting the context- dependent sentiment of individual words and phrases within a message. 1 Introduction The SemEval-2013 task on “Sentiment Analysis in Twitter” (Wilson et al., 2013) focuses on polarity clas- sification, i. e. the problem of determining whether a textual unit, e. g. a document, paragraph, sentence or phrase, expresses a positive, negative or neutral sentiment (for a review of research topics and re- cent developments in the field of sentiment analysis see Liu (2012)). There are two subtasks: in task B, “Message Polarity Classification”, whole messages have to be classified as being of positive, negative or neutral sentiment; in task A, “Contextual Polarity Disambiguation”, a marked instance of a word or phrase has to be classified in the context of a whole message. The training data for task B consist of approxi- mately 10 200 manually annotated Twitter messages, the training data for task A of approximately 9 500 marked instances in approximately 6 300 Twitter mes- sages.1 The test data consist of in-domain Twit- ter messages (3 813 messages for task B and 4 435 marked instances in 2 826 messages for task A) and out-of-domain SMS text messages (2 094 messages for task B, 2 334 marked instances in 1 437 messages for task A). The distribution of messages and marked instances over sentiment categories in the training and test sets is shown in Tab. 1. pos neg neu total train-B 3 783 1 600 4 832 10 215 test-B Twitter 1 572 601 1 640 3 813 test-B SMS 492 394 1 208 2 094 train-A 5 862 3 166 463 9 491 test-A Twitter 2 734 1 541 160 4 435 test-A SMS 1 071 1 104 159 2 334 Table 1: The data sets for both tasks The main focus of the current paper lies on experi- menting with resource-lean and robust methods for task B, the classification of whole messages. We do, however, apply our approach also to task A. 2 Features used for polarity classification Our general approach is quite simple: …

Content truncated to 500 words