Cross-Domain Label-Adaptive Stance Detection

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9011–9028 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9011 Cross-Domain Label-Adaptive Stance Detection Momchil Hardalov1,2 Arnav Arora1,3 Preslav Nakov1,4 Isabelle Augenstein1,3 1Checkstep Research 2Sofia University “St. Kliment Ohridski”, Bulgaria 3University of Copenhagen, Denmark 4Qatar Computing Research Institute, HBKU, Doha, Qatar {momchil, arnav, preslav.nakov, isabelle}@checkstep.com Abstract Stance detection concerns the classification of a writer’s viewpoint towards a target. There are different task variants, e.g., stance of a tweet vs. a full article, or stance with respect to a claim vs. an (implicit) topic. Moreover, task definitions vary, which includes the la- bel inventory, the data collection, and the an- notation protocol. All these aspects hinder cross-domain studies, as they require changes to standard domain adaptation approaches. In this paper, we perform an in-depth analysis of 16 stance detection datasets, and we ex- plore the possibility for cross-domain learn- ing from them. Moreover, we propose an end-to-end unsupervised framework for out- of-domain prediction of unseen, user-defined labels. In particular, we combine domain adap- tation techniques such as mixture of experts and domain-adversarial training with label em- beddings, and we demonstrate sizable perfor- mance gains over strong baselines, both (i) in- domain, i.e., for seen targets, and (ii) out-of- domain, i.e., for unseen targets. Finally, we perform an exhaustive analysis of the cross- domain results, and we highlight the important factors influencing the model performance. 1 Introduction There are many different scenarios in which it is useful to study the attitude expressed in texts, e.g., of politicians with respect to newly proposed legislation (Somasundaran and Wiebe, 2010), of customers regarding new products (Somasundaran and Wiebe, 2009), or of the general public towards public health measures, e.g., aiming to reduce the spread of COVID-19 (Hossain et al., 2020; Glandt et al., 2021). This task, commonly referred to as stance detection, has been studied in many different forms: not just for different domains, but with more substantial differences in the settings, e.g., stance (i) expressed in tweets (Qazvinian et al., 2011; Mohammad et al., 2016; Conforti et al., 2020b) vs. long news articles (Pomerleau and Rao, 2017; Ferreira and Vlachos, 2016) vs. news outlets (Ste- fanov et al., 2020) vs. people (Darwish et al., 2020), (ii) with respect to a claim (Chen et al., 2019) vs. a topic, either explicit (Qazvinian et al., 2011; Walker et al., 2012) or implicit (Hasan and Ng, 2013; Gor- rell et al., 2019). Moreover, there is substantial variation in (iii) the label inventory, in the exact label definition, in the data collection, in the anno- tation setup, in the domain, etc. The most crucial of these, which has not been investigated, currently preventing cross-domain studies, is that the label inventories differ between the settings, as shown in Table 1. Labels include not only variants of agree, disagree, and unrelated, but also difficult to cross-map ones, such as discuss and question. Our goal in this paper is to design a common stance detection framework to facilitate …

Content truncated to 500 words