USFD at SemEval-2016 Task 6: Any-Target Stance Detection on Twitter with Autoencoders

Proceedings of SemEval-2016, pages 389–393, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics USFD at SemEval-2016 Task 6: Any-Target Stance Detection on Twitter with Autoencoders Isabelle Augenstein and Andreas Vlachos and Kalina Bontcheva Department of Computer Science University of Sheffield i.augenstein@sheffield.ac.uk, a.vlachos@sheffield.ac.uk, k.bontcheva@sheffield.ac.uk Abstract This paper describes the University of Sheffield’s submission to the SemEval 2016 Twitter Stance Detection weakly supervised task (SemEval 2016 Task 6, Subtask B). In stance detection, the goal is to classify the stance of a tweet towards a target as “favor”, “against”, or “none”. In Subtask B, the targets in the test data are different from the targets in the training data, thus rendering the task more challenging but also more realistic. To address the lack of target-specific training data, we use a large set of unlabelled tweets containing all targets and train a bag-of-words autoencoder to learn how to produce feature representa- tions of tweets. These feature representations are then used to train a logistic regression clas- sifier on labelled tweets, with additional fea- tures such as an indicator of whether the tar- get is contained in the tweet. Our submitted run on the test data achieved an F1 of 0.3270. 1 Introduction Stance detection is the task of assigning stance la- bels to a piece of text with respect to a topic, i.e. whether a piece of text is in favour of “abortion”, neutral, or against. Previous work considered target- specific stance predictors in debates (Walker et al., 2012; Hasan and Ng, 2013) or news (Ferreira and Vlachos, 2016). The variety of topics discussed on Twitter calls for developing methods that can generalise to any target, including targets not seen in the training data, which is the focus of Subtask B in Task 6 of SemEval 2016 (Mohammad et al., 2016). A further challenge is that the targets are not always mentioned in the tweets, which distinguishes this task from target-dependent sentiment analysis (Zhang et al., 2015; Zhang et al., 2016), and open-domain target-dependent sentiment analysis (Mitchell et al., 2013; Vo and Zhang, 2015). The SemEval Stance Detection task is further re- lated to that of textual entailment (Dagan et al., 2005; Bowman et al., 2015; Lendvai et al., 2016), i.e. we judge if a hypothesis (tweet in our task) entails, contradicts or is neutral towards a textual premise (target in our task). However, the premises in typical RTE datasets offer a richer context than the stance detection targets, i.e. they are full sentences instead of topic labels such as “atheism”. Simple baselines such as textual overlap can achieve an F1 of >0.5 (Bowman et al., 2015), whereas for stance detection such baselines would not perform well, as the target is only mentioned in about half the tweets. In our approach we learn a 3-way logistic regres- sion classifier to perform stance detection. Apart from the standard bag-of-words features commonly used in sentiment analysis, we also use features from a trained bag-of-words autoencoder similar to the one used by Glorot …

Content truncated to 500 words