UKPDIPF: Lexical Semantic Approach to Sentiment Polarity Prediction in Twitter Data

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 704–710, Dublin, Ireland, August 23-24, 2014. UKPDIPF: A Lexical Semantic Approach to Sentiment Polarity Prediction in Twitter Data Lucie Flekova†‡, Oliver Ferschke†‡ and Iryna Gurevych†‡ † Ubiquitous Knowledge Processing Lab (UKP-TUDA) Computer Science Department, Technische Universit¨at Darmstadt ‡ Ubiquitous Knowledge Processing Lab (UKP-DIPF) German Institute for Educational Research http://www.ukp.tu-darmstadt.de Abstract We present a sentiment classification sys- tem that participated in the SemEval 2014 shared task on sentiment analysis in Twit- ter. Our system expands tokens in a tweet with semantically similar expressions us- ing a large novel distributional thesaurus and calculates the semantic relatedness of the expanded tweets to word lists repre- senting positive and negative sentiment. This approach helps to assess the polarity of tweets that do not directly contain po- larity cues. Moreover, we incorporate syn- tactic, lexical and surface sentiment fea- tures. On the message level, our system achieved the 8th place in terms of macro- averaged F-score among 50 systems, with particularly good performance on the Life- Journal corpus (F1=71.92) and the Twitter sarcasm (F1=54.59) dataset. On the ex- pression level, our system ranked 14 out of 27 systems, based on macro-averaged F-score. 1 Introduction Microblogging sites, such as Twitter, have become an important source of information about current events. The fact that users write about their ex- periences, often directly during or shortly after an event, contributes to the high level of emo- tions in many such messages. Being able to auto- matically and reliably evaluate these emotions in context of a specific event or a product would be highly beneficial not only in marketing (Jansen et al., 2009) or public relations, but also in political sciences (O’Connor et al., 2010), disaster manage- This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence details: http://creativecommons.org/licenses/by/4.0/ ment, stock market analysis (Bollen et al., 2011) or the health sector (Culotta, 2010). Due to its large number of applications, senti- ment analysis on Twitter is a very popular task. Challenges arise both from the character of the task and from the language specifics of Twit- ter messages. Messages are normally very short and informal, frequently using slang, alternative spelling, neologism and links, and mostly ignor- ing the punctuation. Our experiments have been carried out as part of the SemEval 2014 Task 9 - Sentiment Anal- ysis on Twitter (Rosenthal et al., 2014), a rerun of a SemEval-2013 Task 2 (Nakov et al., 2013). The datasets are thus described in detail in the overview papers. The rerun uses the same train- ing and development data, but new test data from Twitter and a “surprise domain”. The task con- sists of two subtasks: an expression-level subtask (Subtask A) and a message-level subtask (Subtask B). In subtask A, each tweet in a corpus contained a marked instance of a word or phrase. The goal is to determine whether that instance is positive, negative or …

Content truncated to 500 words