OpenStance: Real-world Zero-shot Stance Detection

Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 314 - 324 December 7-8, 2022 ©2022 Association for Computational Linguistics OpenStance: Real-world Zero-shot Stance Detection Hanzi Xu, Slobodan Vucetic and Wenpeng Yin Temple University {hanzi.xu; slobodan.vucetic; wenpeng.yin}@temple.edu Abstract Prior studies of zero-shot stance detection iden- tify the attitude of texts towards unseen topics occurring in the same document corpus. Such task formulation has three limitations: (i) Sin- gle domain/dataset. A system is optimized on a particular dataset from a single domain; there- fore, the resulting system cannot work well on other datasets; (ii) the model is evaluated on a limited number of unseen topics; (iii) it is assumed that part of the topics has rich annota- tions, which might be impossible in real-world applications. These drawbacks will lead to an impractical stance detection system that fails to generalize to open domains and open-form topics. This work defines OpenStance: open- domain zero-shot stance detection, aiming to handle stance detection in an open world with neither domain constraints nor topic- specific annotations. The key challenge of OpenStance lies in the open-domain gener- alization: learning a system with fully unspe- cific supervision but capable of generalizing to any dataset. To solve OpenStance, we pro- pose to combine indirect supervision, from textual entailment datasets, and weak supervi- sion, from data generated automatically by pre- trained Language Models. Our single system, without any topic-specific supervision, outper- forms the supervised method on three popular datasets. To our knowledge, this is the first work that studies stance detection under the open-domain zero-shot setting. All data and code are publicly released.1 1 Introduction Stance detection differentiates the attitude (e.g., support, oppose, or neutral) of a text to- wards a topic (Walker et al., 2012a). The topic can be a phrase or a complete sentence. The same text can express the author’s positions on many differ- ent topics. For example, a tweet on climate warm- 1https://github.com/xhz0809/OpenStance ing may also express attitudes about environmental policies as well as the debate between electric or fuel cars. Such compound expression can be seen on all online platforms, including News outlets, Twitter, blogs, etc. Therefore, stance detection can be a complicated task that is essential for develop- ing the inference capability of NLP models as well as other disciplines such as politics, journalism, etc. Since the textual expressions and the size of top- ics in the real world are unpredictable, zero-shot stance detection has become the mainstream re- search direction in this area: topics in the test set are unseen during training. For example, Moham- mad et al. (2016) created a dataset SemT6 based on tweets with six noun phrases as topics. One of the topics was reserved for testing and the remaining were used for training. Allaway and McKeown (2020) extended the topic size on the domain of news comments by covering 4,000 topics in train- ing and 600 unseen topics in testing. However, despite the change in the domain and topic size, there are three major limitations in …

Content truncated to 500 words