DeepStance at SemEval-2016 Task 6: Detecting Stance in Tweets Using Character and Word-Level CNNs

Proceedings of SemEval-2016, pages 413–419, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics DeepStance at SemEval-2016 Task 6: Detecting Stance in Tweets Using Character and Word-Level CNNs Prashanth Vijayaraghavan, Ivan Sysoev, Soroush Vosoughi and Deb Roy MIT Media Lab, Massachusetts Institute of Technology Cambridge, MA 02139 pralav@mit.edu, isysoev@mit.edu, soroush@mit.edu, dkroy@media.mit.edu Abstract This paper describes our approach for the De- tecting Stance in Tweets task (SemEval-2016 Task 6). We utilized recent advances in short text categorization using deep learning to cre- ate word-level and character-level models. The choice between word-level and character- level models in each particular case was in- formed through validation performance. Our final system is a combination of classifiers us- ing word-level or character-level models. We also employed novel data augmentation tech- niques to expand and diversify our training dataset, thus making our system more robust. Our system achieved a macro-average preci- sion, recall and F1-scores of 0.67, 0.61 and 0.635 respectively. 1 Introduction Stance detection is the task of automatically deter- mining whether the authors of a text are against or in favour of a given target. For instance, take the fol- lowing sentence: ”It has been such a cold April, so much for global warming.” This sentence’s author is most likely against the concept of global warm- ing (i.e., does not believe in it). The work presented here is specifically targeted towards detecting stance in tweets. The noisy and idiosyncratic nature of tweets make this a particularly hard task. Automatic identification of stance in tweets has practical applications for a range of domains. For instance, it can be used as a sensor to measure the attitude of Twitter users on various issues, such as: political issues, candidates, brand names, TV shows, etc. There has been extensive research done on mod- elling and automatic detection of stance in politi- cal arenas (e.g., debates) (Thomas et al., 2006) and on online forums (Somasundaran and Wiebe, 2009; Murakami and Raymond, 2010). However, as we alluded to earlier, the peculiar nature of tweets make techniques that have been developed for other plat- forms unsuitable. The field closest to this work is the field of Twitter sentiment classification, where the task is to detect the sentiment of a given tweet, usually as positive, negative, or neutral. Nonethe- less, it is important to note that there are substan- tial differences between sentiment classification and stance detection. Sentiment classifiers determine the polarity of a given tweet, without considering any targets (see Vosoughi et al. (Vosoughi et al., 2015) for an example of a Twitter sentiment classi- fier). For instance, consider the tweet: ”I love Don- ald Trump”, this tweet has a positive sentiment, and the author of the tweet has a positive stance towards Donald Trump, but it can also be inferred that the author is most likely against or at best neutral to- wards Bernie Sanders. In this paper, we present a system for automatic detection of stance in Tweets. 2 Our Approach We trained a different model for …

Content truncated to 500 words