SSR: Utilizing Simplified Stance Reasoning Process for Robust Stance Detection

Proceedings of the 29th International Conference on Computational Linguistics, pages 6846–6858 October 12–17, 2022. 6846 SSR: Utilizing Simplified Stance Reasoning Process for Robust Stance Detection Jianhua Yuan1, Yanyan Zhao1, Yanyue Lu1, Bing Qin1 2 1Harbin Institute of Technology, Harbin 150001, China 2Pengcheng Laboratory, Shenzhen 518066, China {jhyuan, yyzhao, yylu, qinb}@ir.hit.edu.cn Abstract Dataset bias in stance detection tasks allows models to achieve superior performance with- out using targets (Kaushal et al., 2021). Most existing debiasing methods are task-agnostic, which fail to utilize task knowledge to better discriminate between genuine and bias features. Motivated by how humans tackle stance de- tection tasks, we propose to incorporate the stance reasoning process as task knowledge to assist in learning genuine features and reducing reliance on bias features. The full stance rea- soning process usually involves identifying the span of the mentioned target and correspond- ing opinion expressions, such fine-grained an- notations are hard and expensive to obtain. To alleviate this, we simplify the stance reasoning process to relax the granularity of annotations from token-level to sentence-level, where la- bels for sub-tasks can be easily inferred from existing resources. We further implement those sub-tasks by maximizing mutual information between the texts and the opinioned targets1. To evaluate whether stance detection models truly understand the task from various aspects, we collect and construct a series of new test sets. Our proposed model achieves better per- formance than previous task-agnostic debiasing methods on most of those new test sets while maintaining comparable performances to exist- ing stance detection models. 1 Introduction The task of stance detection aims to predict the stance of the text towards the given target. It is crucial for various downstream tasks including fact verification, rumor detection, etc. It has a wide ap- plication in analyzing political opinions and prod- uct reviews. Existing works usually treat this task as a text pair classification problem and many de- sign target-tweet interaction structures (Augenstein et al., 2016) to learn target-aware stance representa- tions. However, Kaushal et al. (2021) have shown 1refers to targets that a given tweet expresses opinions on. Tweet: Hilarity of the day: Hillary said she went 'above and beyond' in transparency. Really? What about the 30k deleted emails? #SemST Given Target: Hillary Clinton Gold Stance: Against Target 1: Hillary Clinton Target 2: Feminist Movement Target 3: Atheism ✅ ❌ ❌ Pred. stance: Against Pred. stance: Against Pred. stance: Against Figure 1: An example illustrating that BERT model does not change predictions based on the target. that those models (Du et al., 2017; Devlin et al., 2019) can achieve superior performances only us- ing the tweet. Those end-to-end stance detection models treat the stance reasoning process as a black box and are prone to rely on bias features in the dataset instead of learning the underlying task. For instance, in Figure 1, the BERT model still pre- dicts Against even when the target is changed to an unrelated target like Atheism. Meanwhile, com- mon stance detection models perform poorly on out-of-distribution datasets (Kaushal et …

Content truncated to 500 words