Can Rationalization Improve Robustness?

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3792 - 3805 July 10-15, 2022 ©2022 Association for Computational Linguistics Can Rationalization Improve Robustness? Howard Chen Jacqueline He Karthik Narasimhan Danqi Chen Department of Computer Science, Princeton University {howardchen, karthikn, danqic}@cs.princeton.edu jyh@princeton.edu Abstract A growing line of work has investigated the de- velopment of neural NLP models that can pro- duce rationales—subsets of input that can ex- plain their model predictions. In this paper, we ask whether such rationale models can provide robustness to adversarial attacks in addition to their interpretable nature. Since these models need to first generate rationales (“rationalizer”) before making predictions (“predictor”), they have the potential to ignore noise or adversari- ally added text by simply masking it out of the generated rationale. To this end, we systemat- ically generate various types of ‘AddText’ at- tacks for both token and sentence-level ratio- nalization tasks and perform an extensive em- pirical evaluation of state-of-the-art rationale models across five different tasks. Our ex- periments reveal that rationale models show promise in improving robustness but struggle in certain scenarios—e.g., when the rational- izer is sensitive to position bias or lexical choices of the attack text. Further, leverag- ing human rationales as supervision does not always translate to better performance. Our study is a first step towards exploring the inter- play between interpretability and robustness in the rationalize-then-predict framework.1 1 Introduction Rationale models aim to introduce a degree of inter- pretability into neural networks by implicitly bak- ing in explanations for their decisions (Lei et al., 2016; Bastings et al., 2019; Jain et al., 2020). These models are carried out in a two-stage ‘rationalize- then-predict’ framework, where the model first se- lects a subset of the input as a rationale and then makes its final prediction for the task solely us- ing the rationale. A human can then inspect the selected rationale to verify the model’s reasoning 1Our code is publicly available at: https://github. com/princeton-nlp/rationale-robustness. The look is an inviting, deep darkness… The look is an inviting, deep darkness… The tea looks horrible. The look is an inviting, deep darkness… Rationalizer Full-context Model Predictor Positive ✓ Positive ✓ The tea looks horrible. The look is an inviting, deep darkness… Rationalizer Full-context Model Negative ✗ Predictor Positive ✓ Figure 1: Top: input text is processed by a rationale model (rationalizer and predictor) and a full-context model (making predictions based on the whole input) separately in a beer review sentiment classification dataset. Both models make correct predictions. Bot- tom: when an attack sentence “The tea looks horrible.” is inserted, the full-context model fails. The rational- izer successfully excludes the negative-sentiment word “horrible” from the selected rationales (yellow high- lights). The predictor is hence not distracted by the attack sentence. over the most relevant parts of the input for the prediction at hand. While previous work has mostly focused on the plausibility of extracted rationales and whether they represent faithful explanations (DeYoung et …

Content truncated to 500 words