iLab-Edinburgh at SemEval-2016 Task 7: A Hybrid Approach for Determining Sentiment Intensity of Arabic Twitter Phrases

Proceedings of SemEval-2016, pages 474–480, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics iLab-Edinburgh at SemEval-2016 Task 7: A Hybrid Approach for Determining Sentiment Intensity of Arabic Twitter Phrases Eshrag Refaee and Verena Rieser Interaction Lab, School of Mathematical and Computer Sciences, Heriot-Watt University, EH14 4AS Edinburgh, United Kingdom. eaar1@hw.ac.uk, v.t.rieser@hw.ac.uk Abstract This paper describes the iLab-Edinburgh Sen- timent Analysis system, winner of the Ara- bic Twitter Task 7 in SemEval-2016. The system employs a hybrid approach of super- vised learning and rule-based methods to pre- dict a sentiment intensity (SI) score for a given Arabic Twitter phrase. First, the supervised method uses an ensemble of trained linear re- gression models to produce an initial SI score for each given text instance. Second, the re- sulting SI score is adjusted using a set of rules that exploit a number of publicly available sentiment lexica. The system demonstrates strong results of 0.536 Kendall score, ranking top in this task. 1 Introduction Sentiment Analysis (SA) concerns the automatic ex- traction and classification of sentiment-related in- formation from a given text instance (Thelwall et al., 2012). This is the first time SA on Arabic text is considered in an international competition, like SemEval. Most of previous work on SA is in En- glish, but there have been recent attempts to address SA for Arabic, e.g. (Abdul-Mageed et al., 2012; Mourad and Darwish, 2013; Refaee and Rieser, 2014c; Refaee and Rieser, 2014b). Previous work in this area has mainly focused on identifying the sentiment polarity in a given tweet/phrase, whereas within SemEval-2016 Task 7, the task is to predict the Sentiment Intensity (SI) in Arabic tweets. That is, in addition to their prior association to a senti- ment class, i.e. positive or negative, each text in- stance has an SI score that indicates the strength of its assigned sentiment on a scale from 0 to 1. In this work, we use a combination of super- vised learning and rule-based approaches, exploit- ing a number of publicly available sentiment lex- ica. We find that the quality (rather than quantity) of these lexica influence system performance for the supervised part of the system. Our best performing system demonstrates strong results of 0.536 Kendall score, ranking top in SemEval-2016 Task 7. This type of hybrid approach between rule-based and sta- tistical methods has been demonstrated to be suc- cessful in other shared tasks, such as dialogue state tracking (Wang and Lemon, 2013). 2 Related Work Research on predicting Sentiment Intensity in Ara- bic is still limited. For example, El-Beltagy and Ali (2013) built a sentiment lexicon in which each entry is manually assigned an SI score. Using this lexi- con, they calculated the overall Sentiment Orienta- tion for a set of Egyptian tweets by adding up the score of extracted positive/negative words. The au- thors observed a significant improvement of up to 20.6% in accuracy when exploiting the SI scores, as compared to results using a uniform weighting scheme, i.e. positive word= +1 and …

Content truncated to 500 words