KACE: Generating Knowledge Aware Contrastive Explanations for Natural Language Inference

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 2516–2527 August 1–6, 2021. ©2021 Association for Computational Linguistics 2516 KACE: Generating Knowledge-Aware Contrastive Explanations for Natural Language Inference Qianglong Chen1,2∗, Feng Ji3†, Xiangji Zeng1, Feng-Lin Li2, Ji Zhang2, Haiqing Chen2, Yin Zhang1‡ 1College of Computer Science and Technology, Zhejiang University, China 2DAMO Academy, Alibaba Group, China, 3Tencent, China {chenqianglong,zengxiangji,zhangyin98}@zju.edu.cn {fenglin.lfl,zj122146,haiqing.chenhq}@alibaba-inc.com {neilji}@tencent.com Abstract In order to better understand the reason be- hind model behaviors (i.e., making predic- tions), most recent work has exploited gener- ative models to provide complementary expla- nations. However, existing approaches in nat- ural language processing (NLP) mainly focus on “WHY A” rather than contrastive “WHY A NOT B”, which is shown to be able to better distinguish confusing candidates and improve model performance in other research fields. In this paper, we focus on generating Contrastive Explanations with counterfactual examples in NLI and propose a novel Knowledge-Aware generation framework (KACE). Specifically, we first identify rationales (i.e., key phrases) from input sentences, and use them as key perturbations for generating counterfactual ex- amples. After obtaining qualified counterfac- tual examples, we take them along with orig- inal examples and external knowledge as in- put, and employ a knowledge-aware genera- tive pre-trained language model to generate contrastive explanations. Experimental results show that contrastive explanations are bene- ficial to clarify the difference between pre- dicted answer and other answer options. More- over, we train an BERT-large based NLI model enhanced with contrastive explanations and achieve an accuracy of 91.9% on SNLI, gain- ing an improvement of 5.7% against ETPA (“Explain-Then-Predict-Attention”) and 0.6% against NILE (“WHY A”). 1 Introduction In recent years, pre-trained language models (De- vlin et al., 2019; Liu et al., 2019; Yang et al., 2019) have been widely adopted in many tasks of natu- ral language processing (Talmor et al., 2019; Choi et al., 2018; Bowman et al., 2015). However, due to ∗ Work is done during internship at Alibaba Group. † The work is mainly conducted while being at Alibaba Group. ‡ Corresponding Author: Yin Zhang the lack of textual explanations, most downstream models become more complicated and difficult to understand. End users, especially those working in critical domains such as healthcare or online ed- ucation, become more skeptical and reluctant to adopt or trust them, although these models have been proved to improve the decision-making per- formance. Therefore, providing faithful textual explanations has become a promising way to over- come the black-box property of neural networks, which has attracted the attention of academia and industrial communities. Recently, the majority of existing methods (Xu et al., 2020; Cheng et al., 2020; Karimi et al., 2020; Ramamurthy et al., 2020; Atanasova et al., 2020; Kumar and Talukdar, 2020) in natural language processing try to explain the predictions of neu- ral models in a model-intrinsic or model-agnostic (also known as post-hoc) way. While post-hoc models (Chen et al., 2020b; Karimi et al., 2020; Kumar and Talukdar, 2020) …

Content truncated to 500 words