MITRE at SemEval-2016 Task 6: Transfer Learning for Stance Detection

Proceedings of SemEval-2016, pages 458–463, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics MITRE at SemEval-2016 Task 6: Transfer Learning for Stance Detection Guido Zarrella and Amy Marsh The MITRE Corporation 202 Burlington Road Bedford, MA 01730-1420, USA jzarrella,amarsh@mitre.org Abstract We describe MITRE’s submission to the SemEval-2016 Task 6, Detecting Stance in Tweets. This effort achieved the top score in Task A on supervised stance detection, pro- ducing an average F1 score of 67.8 when as- sessing whether a tweet author was in favor or against a topic. We employed a recurrent neu- ral network initialized with features learned via distant supervision on two large unlabeled datasets. We trained embeddings of words and phrases with the word2vec skip-gram method, then used those features to learn sentence rep- resentations via a hashtag prediction auxiliary task. These sentence vectors were then fine- tuned for stance detection on several hundred labeled examples. The result was a high per- forming system that used transfer learning to maximize the value of the available training data. 1 Introduction This paper describes a system for performing au- tomatic stance detection in social media messages. Our approach employs a recurrent neural network which was initialized from pre-trained features learned in successive attempts to encode world knowledge via weak external supervision. Stance detection is the task of determining whether the author of a text is in favor or against a given topic, while rejecting texts in which neither inference is likely. This task is distinct from senti- ment analysis in that an in favor or against stance can be measured independently of an author’s emo- tional state. In stance detection we attempt to mea- sure how an author’s opinion is expressed in sponta- neous, unstructured messages rather than the explicit prompts of formal opinion polls. Declarations of stance are often couched in fig- urative language that can be difficult for machines to unravel. Consider the texts We don’t inherit the earth from our parents we borrow it from our chil- dren and Last time I checked, Al Gore is a politi- cian, not a scientist. To the human observer mes- sages like these contain an interpretable stance rel- evant to the topic of climate change. But to under- stand rhetorical devices like sarcasm, irony, analogy, and metaphor, a reader often uses personal experi- ence to infer broader context. For machines, matters are additionally complicated by use of informal vo- cabulary, grammar, and spelling. Furthermore, train- ing data is often expensive or difficult to collect in bulk. These challenges motivated our efforts to seek transfer learning of broad world knowledge through feature pre-training using large unlabeled datasets. 2 Related Work It is common for machine learning approaches to be- gin learning of any new task from scratch, for exam- ple by randomly initializing the parameters of a neu- ral network. This disregards any knowledge gained by similar algorithms when solving previous tasks. Transfer learning approaches, on the other hand, store the knowledge gained in one context and …

Content truncated to 500 words