Incorporating Priors with Feature Attribution on Text Classification

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6274–6283 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 6274 Incorporating Priors with Feature Attribution on Text Classification Frederick Liu GoogleBesim Avci {frederickliu, besim}@google.com Abstract Feature attribution methods, proposed re- cently, help users interpret the predictions of complex models. Our approach integrates fea- ture attributions into the objective function to allow machine learning practitioners to incor- porate priors in model building. To demon- strate the effectiveness our technique, we ap- ply it to two tasks: (1) mitigating unintended bias in text classifiers by neutralizing identity terms; (2) improving classifier performance in a scarce data setting by forcing the model to focus on toxic terms. Our approach adds an L2 distance loss between feature attributions and task-specific prior values to the objective. Our experiments show that i) a classifier trained with our technique reduces undesired model biases without a tradeoff on the original task; ii) incorporating priors helps model perfor- mance in scarce data settings. 1 Introduction One of the recent challenges in machine learn- ing (ML) is interpreting the predictions made by models, especially deep neural networks. Under- standing models is not only beneficial, but nec- essary for wide-spread adoption of more complex (and potentially more accurate) ML models. From healthcare to financial domains, regulatory agen- cies mandate entities to provide explanations for their decisions (Goodman and Flaxman, 2016). Hence, most machine learning progress made in those areas is hindered by a lack of model explain- ability – causing practitioners to resort to simpler, potentially low-performance models. To supply for this demand, there has been many attempts for model interpretation in recent years for tree-based algorithms (Lundberg et al., 2018) and deep learn- ing algorithms (Lundberg and Lee, 2017; Smilkov et al., 2017; Sundararajan et al., 2017; Bach et al., 2015; Kim et al., 2018; Dhurandhar et al., 2018). Method Sentence Probability Baseline I am gay 0.915 I am straight 0.085 Our Method I am gay 0.141 I am straight 0.144 Table 1: Toxicity probabilities for samples of a base- line CNN model and our proposed method. Words are shaded based on their attribution and italicized if attri- bution is > 0. On the other hand, the amount of research fo- cusing on explainable natural language process- ing (NLP) models (Li et al., 2016; Murdoch et al., 2018; Lei et al., 2016) is modest as opposed to im- age explanation techniques. Inherent problems in data emerge in a trained model in several ways. Model explanations can show that the model is not inline with human judg- ment or domain expertise. A canonical example is model unfairness, which stems from biases in the training data. Fairness in ML models rightfully came under heavy scrutiny in recent years (Zhang et al., 2018a; Dixon et al., 2018; Angwin et al., 2016). Some examples include sentiment analysis models weighing negatively for inputs containing identity terms such as “jew” and “black”, and hate speech classifiers leaning to …

Content truncated to 500 words