A Fine-grained Interpretability Evaluation Benchmark for Neural NLP

Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 70 - 84 December 7-8, 2022 ©2022 Association for Computational Linguistics A Fine-grained Interpretability Evaluation Benchmark for Neural NLP Lijie Wang, Yaozong Shen, Shuyuan Peng, Shuai Zhang, Xinyan Xiao, Hao Liu, Hongxuan Tang, Ying Chen, Hua Wu, Haifeng Wang Baidu Inc, Beijing, China {wanglijie,shenyaozong@baidu.com} Abstract While there is increasing concern about the in- terpretability of neural models, the evaluation of interpretability remains an open problem, due to the lack of proper evaluation datasets and metrics. In this paper, we present a novel bench- mark to evaluate the interpretability of both neu- ral models and saliency methods. This bench- mark covers three representative NLP tasks: sentiment analysis, textual similarity and read- ing comprehension, each provided with both English and Chinese annotated data. In order to precisely evaluate the interpretability, we provide token-level rationales that are carefully annotated to be sufficient, compact and compre- hensive. We also design a new metric, i.e., the consistency between the rationales before and after perturbations, to uniformly evaluate the in- terpretability on different types of tasks. Based on this benchmark, we conduct experiments on three typical models with three saliency meth- ods, and unveil their strengths and weakness in terms of interpretability. We will release this benchmark1 and hope it can facilitate the research in building trustworthy systems. 1 Introduction In the last decade, deep learning (DL) has been rapidly developed and has greatly improved vari- ous artificial intelligence tasks in terms of accuracy (Deng and Yu, 2014; Litjens et al., 2017; Pouyanfar et al., 2018). However, as DL models are black-box systems, their inner decision processes are opaque to users. This lack of transparency makes them untrustworthy and hard to be applied in decision- making applications in fields such as health, com- merce and law (Fort and Couillault, 2016). Conse- quently, there is a growing interest in explaining the predictions of DL models (Simonyan et al., 2014; Ribeiro et al., 2016; Alzantot et al., 2018; Bastings et al., 2019; Jiang et al., 2021). Accordingly, many 1https://www.luge.ai/#/luge/task/taskDetail? taskId=15 Sentiment Analysis (SA) Instanceo: although it bangs a very cliched drum at times, this crowd-pleaser’s fresh dialogue, energetic music, and good-natured spunk are often infectious. Sentiment label: positive Instancep: although it bangs a very cliched drum at times, this crowd-pleaser’s novel dialogue, vigorous music, and good-natured spunk are often infectious. Sentiment label: positive Semantic Textual Similarity (STS) Instance1o: Is there a reason why we should travel alone? Instance2o: What are some reasons to travel alone? Similarity: same Instance1p: Is there any reason why we travel alone? Instance2p: List some reasons to travel alone? Similarity: same Machine Reading Comprehensive (MRC) Question: What part of France were the Normans located? Articleo: ...and customs to synthesize a unique “Norman” culture in the north of France. ... Answer: north Question: Where in France were the Normans located? Articlep: ...and customs to synthesize a unique “Norman” culture in the north of France. ... Answer: north Table 1: Examples from our benchmark. …

Content truncated to 500 words