Measuring Association Between Labels and Free-Text Rationales

Measuring Association Between Labels and Free-Text Rationales Sarah Wiegreffe♣ Ana Marasovi´c†3 Noah A. Smith†3 ♣School of Interactive Computing, Georgia Institute of Technology †Allen Institute for Artificial Intelligence 3Paul G. Allen School of Computer Science and Engineering, University of Washington saw@gatech.edu, {anam,noah}@allenai.org Abstract In interpretable NLP, we require faithful ratio- nales that reflect the model’s decision-making process for an explained instance. While prior work focuses on extractive rationales (a subset of the input words), we investigate their less- studied counterpart: free-text natural language rationales. We demonstrate that pipelines, mod- els for faithful rationalization on information- extraction style tasks, do not work as well on “reasoning” tasks requiring free-text rationales. We turn to models that jointly predict and ratio- nalize, a class of widely used high-performance models for free-text rationalization. We inves- tigate the extent to which the labels and ratio- nales predicted by these models are associated, a necessary property of faithful explanation. Via two tests, robustness equivalence and fea- ture importance agreement, we find that state- of-the-art T5-based joint models exhibit desir- able properties for explaining commonsense question-answering and natural language infer- ence, indicating their potential for producing faithful free-text rationales.1 1 Introduction Interpretable NLP aims to better understand pre- dictive models’ internals for purposes such as de- bugging, validating safety before deployment, or revealing unintended biases and behavior (Molnar, 2019). These objectives require faithful rationales— explanations of the model’s behavior that are accu- rate representations of its decision process (Melis and Jaakkola, 2018). One way towards faithfulness is to introduce architectural modifications or constraints that pro- duce rationales with desirable properties (Andreas et al., 2016; Schwartz et al., 2018; Jiang et al., 2019, inter alia). For example, pipeline models (Figure 2) were designed for information extraction (IE) tasks for which a rationale can be extracted as a subset of 1Our code is available at https://github.com/ allenai/label_rationale_association. Figure 1: A categorization of interpretable NLP on an illustrative faithfulness spectrum. Two predominant forms of explanation exist that align with two predomi- nant classes of NLP tasks. Unlike models for IE tasks, the desirable properties of interpretable models for rea- soning tasks have not been explored. We investigate architectures and tests for explaining reasoning tasks. the input and is sufficient to make a prediction on its own, without the rest of the input (Lei et al., 2016). Such models approach faithfulness by construction (Jain et al., 2020). There is a growing interest in tasks that require world and commonsense “knowledge” and “rea- soning”, such as commonsense question-answering (CommonsenseQA; Talmor et al., 2019) and nat- ural language inference (SNLI; Bowman et al., 2015). Here, extractive rationales necessarily fall short—rationales must instead take the form of free-text natural language to fill in the reasoning or knowledge gap (Camburu et al., 2018; Rajani et al., 2019).2 In Table 1, for example, the highlighted ex- tractive rationale of the first problem instance lacks at least one reasoning step to adequately justify the answer; the natural language rationale (which is not extractive) fills in the gap. We …

Content truncated to 500 words