NILE : Natural Language Inference with Faithful Natural Language Explanations

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8730–8742 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 8730 NILE : Natural Language Inference with Faithful Natural Language Explanations Sawan Kumar Indian Institute of Science, Bangalore sawankumar@iisc.ac.in Partha Talukdar Indian Institute of Science, Bangalore ppt@iisc.ac.in Abstract The recent growth in the popularity and suc- cess of deep learning models on NLP classi- fication tasks has accompanied the need for generating some form of natural language ex- planation of the predicted labels. Such gen- erated natural language (NL) explanations are expected to be faithful, i.e., they should cor- relate well with the model’s internal decision making. In this work, we focus on the task of natural language inference (NLI) and ad- dress the following question: can we build NLI systems which produce labels with high accu- racy, while also generating faithful explana- tions of its decisions? We propose Natural- language Inference over Label-specific Expla- nations (NILE), a novel NLI method which uti- lizes auto-generated label-specific NL expla- nations to produce labels along with its faith- ful explanation. We demonstrate NILE’s ef- fectiveness over previously reported methods through automated and human evaluation of the produced labels and explanations. Our evaluation of NILE also supports the claim that accurate systems capable of providing testable explanations of their decisions can be designed. We discuss the faithfulness of NILE’s explanations in terms of sensitivity of the decisions to the corresponding explana- tions. We argue that explicit evaluation of faithfulness, in addition to label and explana- tion accuracy, is an important step in evaluat- ing model’s explanations. Further, we demon- strate that task-specific probes are necessary to establish such sensitivity. 1 Introduction Deep learning methods have been employed to im- prove performance on several benchmark classi- fication tasks in NLP (Wang et al., 2018, 2019). Typically, these models aim at improving label ac- curacy, while it is often desirable to also produce explanations for these decisions (Lipton, 2016; Chakraborty et al., 2017). In this work, we focus on producing natural language explanations for Natu- ral Language Inference (NLI), without sacrificing much on label accuracy. There has been growing interest in producing natural language explanations for deep learning systems (Huk Park et al., 2018; Kim et al., 2018; Ling et al., 2017), including NLI (Camburu et al., 2018). In general, the explanations from these methods can typically be categorized as post-hoc explanations (Lipton, 2016). Camburu et al. (2018) propose an NLI system which first produces an explanation and then processes the explanation to produce the final label. We argue that these expla- nations also resemble post-hoc explanations (Sec- tion 4.2). Further, existing methods don’t provide a natural way to test the faithfulness of the gener- ated explanations, i.e., how well do the provided explanations correlate with the model’s decision making. We therefore propose Natural-language Infer- ence over Label-specific Explanations (NILE)1, which we train and evaluate on English language examples. Through NILE, we aim to answer the following question: Can we build …

Content truncated to 500 words