QCRI at SemEval-2016 Task 4: Probabilistic Methods for Binary and Ordinal Quantification

Proceedings of SemEval-2016, pages 58–63, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics QCRI at SemEval-2016 Task 4: Probabilistic Methods for Binary and Ordinal Quantification Giovanni Da San Martino, Wei Gao, Fabrizio Sebastiani∗ Qatar Computing Research Institute Hamad bin Khalifa University PO Box 5825, Doha, Qatar Email: {gmartino,wgao,fsebastiani}@qf.org.qa Abstract We describe the systems we have used for participating in Subtasks D (binary quan- tification) and E (ordinal quantification) of SemEval-2016 Task 4 “Sentiment Analysis in Twitter”. The binary quantification sys- tem uses a “Probabilistic Classify and Count” (PCC) approach that leverages the calibrated probabilities obtained from the output of an SVM. The ordinal quantification approach uses an ordinal tree of PCC binary quantifiers, where the tree is generated via a splitting crite- rion that minimizes the ordinal quantification loss. 1 Introduction This document describes the systems we have used for participating in Subtasks D (binary quantifica- tion) and E (ordinal quantification) of SemEval- 2016 Task 4 “Sentiment Analysis in Twitter”. In the runs we have submitted no training data was used other than the officially provided ones (indeed, the only “external” data used were the sentiment lexi- cons mentioned in Section 2). Like a classification system, a system for per- forming quantification consists of two main compo- nents: (i) an algorithm for converting the objects of interest (tweets, in our case) into vectorial represen- tations that can be interpreted both by the learning algorithm and, once it has been trained, by the quan- tifier itself, and (ii) an algorithm for training quan- tifiers from vectorial representations of training ob- ∗Fabrizio Sebastiani is currently on leave from Consiglio Nazionale delle Ricerche, Italy. jects. Section 2 will be devoted to discussing com- ponent (i), while Sections 3 and 4 will be devoted to discussing the two learning algorithms we have deployed for the two tasks. 2 Features for detecting tweet sentiment As in (Gao and Sebastiani, 2015; Gao and Sebas- tiani, 2016), for building vectorial representations of tweets we have followed the approach discussed in (Kiritchenko et al., 2014, Section 5.2.1), since the representations presented therein are those used in the systems that performed best at both the Se- mEval 2013 (Mohammad et al., 2013) and SemEval 2014 (Zhu et al., 2014) tweet sentiment classifica- tion shared tasks. The text is preprocessed by normalizing URLs and mentions of users to the constants http://someurl and @someuser, resp., after which tokenisation and POS tagging is performed. The binary features used (i.e., features denoting presence or absence in the tweet) include word n- grams, for n ∈ {1, 2, 3, 4}, and character n-grams, for n ∈ {3, 4, 5}, whether the last token contains an exclamation and/or a question mark, whether the last token is a positive or a negative emoticon and, for each of the 1000 word clusters produced with the CMU Twitter NLP tool1, whether any token from the cluster is present. Integer-valued features include the number of all-caps tokens, the number of tokens for each POS tag, the …

Content truncated to 500 words