Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 8913–8931, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 8913 Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations Emily Allaway Columbia University New York, NY eallaway@cs.columbia.edu Kathleen McKeown Columbia University New York, NY kathy@cs.columbia.edu Abstract Stance detection is an important component of understanding hidden influences in every- day life. Since there are thousands of poten- tial topics to take a stance on, most with lit- tle to no training data, we focus on zero-shot stance detection: classifying stance from no training examples. In this paper, we present a new dataset for zero-shot stance detection that captures a wider range of topics and lex- ical variation than in previous datasets. Addi- tionally, we propose a new model for stance detection that implicitly captures relationships between topics using generalized topic repre- sentations and show that this model improves performance on a number of challenging lin- guistic phenomena. 1 Introduction Stance detection, automatically identifying posi- tions on a specific topic in text (Mohammad et al., 2017), is crucial for understanding how informa- tion is presented in everyday life. For example, a news article on crime may also implicitly take a position on immigration (see Table 1). There are two typical approaches to stance de- tection: topic-specific stance (developing topic- specific classifiers, e.g., Hasan and Ng (2014)) and cross-target stance (adapting classifiers from a re- lated topic to a single new topic detection, e.g., Augenstein et al. (2016)). Topic-specific stance requires the existence of numerous, well-labeled training examples in order to build a classifier for a new topic, an unrealistic expectation when there are thousands of possible topics for which data col- lection and annotation are both time-consuming and expensive. While cross-target stance does not require training examples for a new topic, it does require human knowledge about any new topic and how it is related to the training topics. As a result, models developed for this variation are still limited Topic: immigration Stance: against Text: The jury’s verdict will ensure that another violent criminal alien will be removed from our community for a very long period ... Table 1: Example snippet from Fox News describing a crime but taking a stance against immigration. Phrases indicating stance are highlighted. in their ability to generalize to a wide variety of topics. In this work, we propose two additional varia- tions of stance detection: zero-shot stance detection (a classifier is evaluated on a large number of com- pletely new topics) and few-shot stance detection (a classifier is evaluated on a large number of top- ics for which it has very few training examples). Neither variation requires any human knowledge about the new topics or their relation to training topics. Zero-shot stance detection, in particular, is a more accurate evaluation of a model’s ability to generalize to the range of topics in the real world. Existing stance datasets typically have a small number of topics (e.g., 6) that are described in …

Content truncated to 500 words