Human Rationales as Attribution Priors for Explainable Stance Detection

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5540–5554 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5540 Human Rationales as Attribution Priors for Explainable Stance Detection Sahil Jayaram Columbia University New York, NY sahil.j@columbia.edu Emily Allaway Columbia University New York, NY eallaway@cs.columbia.edu Abstract As NLP systems become better at detecting opinions and beliefs from text, it is important to ensure not only that models are accurate but also that they arrive at their predictions in ways that align with human reasoning. In this work, we present a method for imparting human-like rationalization to a stance detection model us- ing crowdsourced annotations on a small frac- tion of the training data. We show that in a data-scarce setting, our approach can improve the reasoning of a state-of-the-art classifier— particularly for inputs containing challenging phenomena such as sarcasm—at no cost in pre- dictive performance. Furthermore, we demon- strate that attention weights surpass a leading attribution method in providing faithful expla- nations of our model’s predictions, thus serv- ing as a computationally cheap and reliable source of attributions for our model. 1 Introduction Stance detection, automatically identifying the po- sition on a topic taken by a text (Mohammad et al., 2017), allows readers to glean valuable informa- tion from news articles and social media, such as whether the writing is politically slanted. Due to the sensitive nature of many topics (e.g., political ideologies and religious beliefs), it is crucial that stance models are transparent and rationalize their predictions in human-like ways. Furthermore, their reasoning must remain human-like even when they are tasked with generalizing to a new, unseen test topic. A model’s rationale for a specific input can be extracted in the form of feature attributions, which quantify the influence that each input feature asserts on the model’s predictions. Methods for obtaining these attributions vary by faithfulness (the extent to which they accurately measure the importance of each feature). Attributions can be incorporated into the training process via attribution priors, a pow- erful framework for imparting domain knowledge Topic = marijuana True Label = FOR Baseline The better question is, though, how many people who have used marijuana DO NOT use other illegal drugs. I think the answer would surprise you. Correlation is not causation - the sooner you learn that, the sooner you can get to the real root of our opiate ad- dition problem. Prediction AGAINST (wrong) Our Method The better question is, though, how many people who have used marijuana DO NOT use other illegal drugs. I think the answer would surprise you. Cor- relation is not causation - the sooner you learn that, the sooner you can get to the real root of our opiate addition problem. Prediction FOR (correct) Table 1: Model reasoning as explained by mean atten- tion weights (MAW). The baseline is trained using only cross-entropy loss, while the second model is trained using our proposed attribution prior. to models (Erion et al., 2019, 2020). In this work, we propose …

Content truncated to 500 words