Improving Stance Detection with Multi-Dataset Learning and Knowledge Distillation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6332–6345 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6332 Improving Stance Detection with Multi-Dataset Learning and Knowledge Distillation Yingjie Li Chenye Zhao Cornelia Caragea University of Illinois at Chicago {yli300,czhao43,cornelia}@uic.edu Abstract Stance detection determines whether the au- thor of a text is in favor of, against or neu- tral to a specific target and provides valu- able insights into important events such as legalization of abortion. Despite significant progress on this task, one of the remaining challenges is the scarcity of annotations. Be- sides, most previous works focused on a hard- label training in which meaningful similarities among categories are discarded during train- ing. To address these challenges, first, we eval- uate a multi-target and a multi-dataset train- ing settings by training one model on each dataset and datasets of different domains, re- spectively. We show that models can learn more universal representations with respect to targets in these settings. Second, we investi- gate the knowledge distillation in stance de- tection and observe that transferring knowl- edge from a teacher model to a student model can be beneficial in our proposed training set- tings. Moreover, we propose an Adaptive Knowledge Distillation (AKD) method that ap- plies instance-specific temperature scaling to the teacher and student predictions. Results show that the multi-dataset model performs best on all datasets and it can be further im- proved by the proposed AKD, outperforming the state-of-the-art by a large margin. We pub- licly release our code.1 1 Introduction People often express their stances toward specific targets (e.g., political figures, or abortion) on so- cial media. These opinions can provide valuable insights into important events, e.g., legalization of abortion. The goal of stance detection is to deter- mine whether the author of a text is in favor of, against or neutral toward a specific target (Moham- mad et al., 2016b; Küçük and Can, 2020; AlDayel and Magdy, 2020). For example, for the tweet in 1https://github.com/chuchun8/ MDL-Stance-Distillation Tweet: We all have a duty to protect the sanctity of life...from the first cell division, to the last. #Pro- tectLife #pjnet #ctot #ccot #SemST Target: Legalization of Abortion Stance: Against Table 1: An example of stance detection. Table 1, we can infer that the author is against to the legalization of abortion implied by the presence of the words “protect the sanctity of life”. Even though stance detection has received a lot of attention, one of the biggest challenges for the stance detection tasks is the scarcity of annotated data. Even worse, previous studies (Mohammad et al., 2017; Du et al., 2017; Wei et al., 2018; Li and Caragea, 2019; Siddiqua et al., 2019) focused on a per-target training strategy, which aims to train one model for each target and evaluate it on the test data corresponding to that target (which we call ad-hoc training). In this case, the model is more likely to make predictions based on specific words without fully considering the target information, …

Content truncated to 500 words