Comparing Automatic and Human Evaluation of Local Explanations for Text Classification

Proceedings of NAACL-HLT 2018, pages 1069–1078 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Comparing Automatic and Human Evaluation of Local Explanations for Text Classification Dong Nguyen♠♦ ♠The Alan Turing Institute, London ♦School of Informatics, University of Edinburgh, Edinburgh dnguyen@turing.ac.uk Abstract Text classification models are becoming in- creasingly complex and opaque, however for many applications it is essential that the mod- els are interpretable. Recently, a variety of approaches have been proposed for generat- ing local explanations. While robust evalua- tions are needed to drive further progress, so far it is unclear which evaluation approaches are suitable. This paper is a first step towards more robust evaluations of local explanations. We evaluate a variety of local explanation ap- proaches using automatic measures based on word deletion. Furthermore, we show that an evaluation using a crowdsourcing experi- ment correlates moderately with these auto- matic measures and that a variety of other fac- tors also impact the human judgements. 1 Introduction While the impact of machine learning is increasing rapidly in society, machine learning systems have also become increasingly complex and opaque. Classification models are usually evaluated based on prediction performance alone (e.g., by measur- ing the accuracy, recall, and precision) and the in- terpretability of these models has generally been undervalued. However, the importance of inter- pretable models is increasingly being recognized (Doshi-Velez and Kim, 2017; Freitas, 2014). First, higher interpretability could lead to more effective models by revealing incompleteness in the problem formalization (Doshi-Velez and Kim, 2017), by revealing confounding factors that could lead to biased models, and by supporting error analyses or feature discovery (Aubakirova and Bansal, 2016). Second, with the increasing adop- tion of machine learning approaches for humani- ties and social science research, there is also an in- creasing need for systems that support exploratory analyses and theory development. Various approaches have been explored to in- crease the interpretability of machine learning models (Lipton, 2016). This paper focuses on lo- cal explanation, which aims to explain the predic- tion for an individual instance (e.g., Ribeiro et al. (2016)). A study by Herlocker et al. (2000) found that providing local explanations could help im- prove the acceptance of movie recommendation systems. Local explanations can come in different forms. For example, Koh and Liang (2017) iden- tify the most influential training documents for a particular prediction. The most common type of local explanation involves identifying the impor- tant parts of the input for a prediction, such as the most predictive words in a document for a text classification model. In this paper we focus on local explanations for text classification. Below is a fragment of a movie review. The words identified by a local explana- tion method to explain a neural network prediction are in bold. The review is labeled with a negative sentiment, but the classifier incorrectly predicted a positive sentiment. The highlighted words help us understand why. steve martin is one of the funniest men alive. if you can take that as …

Content truncated to 500 words