It Takes Two Flints to Make a Fire: Multitask Learning of Neural Relation and Explanation Classifiers

It Takes Two Flints to Make a Fire: Multitask Learning of Neural Relation and Explanation Classifiers Zheng Tang University of Arizona Department of Computer Science zhengtang@arizona.edu Mihai Surdeanu University of Arizona Department of Computer Science msurdeanu@arizona.edu We propose an explainable approach for relation extraction that mitigates the tension between generalization and explainability by jointly training for the two goals. Our approach uses a multi-task learning architecture, which jointly trains a classifier for relation extraction, and a sequence model that labels words in the context of the relations that explain the decisions of the relation classifier. We also convert the model outputs to rules to bring global explanations to this approach. This sequence model is trained using a hybrid strategy: supervised, when supervision from pre-existing patterns is available, and semi-supervised otherwise. In the latter situation, we treat the sequence model’s labels as latent variables, and learn the best assignment that maximizes the performance of the relation classifier. We evaluate the proposed approach on the two datasets and show that the sequence model provides labels that serve as accurate explanations for the relation classifier’s decisions, and, importantly, that the joint training generally improves the performance of the relation classifier. We also evaluate the performance of the generated rules and show that the new rules are a great add-on to the manual rules and bring the rule-based system much closer to the neural models. 1. Introduction Many domains such as medical, legal, or finance, require that decision making be not only accurate but also trustworthy. Thus, understanding what the underlying model captures is a critical requirement in such applications. To this end, previous efforts addressed this limitation by adding explainability to neural models, which have come to dominate natural language processing (NLP) (Manning 2015). These explanations can be categorized along two main aspects: whether they explain a complete model (global) or individual predictions (local); and whether they are an integral part of the Action Editor: Vivek Srikumar. Submission received: 30 March 2022; revised version received: 31 August 2022; accepted for publication: 9 September 2022. https://doi.org/10.1162/coli a 00463 © 2022 Association for Computational Linguistics Published under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license Computational Linguistics Volume 49, Number 1 classification model itself (self-explaining) or are generated through a post-processing step (post-hoc) (see Section 2 for a longer discussion). Most of the recent proposed efforts focus on the local and post-hoc explanations (Ribeiro, Singh, and Guestrin 2016; Shapley 1952; Schwab and Karlen 2019). These directions have a few advantages such as modularity and simplicity. However, they also have two important drawbacks: These types of explanations are not guaranteed to be faithful to the original model to be explained, and are not actionable, that is, even if they correctly explain an imperfect classification, there is no clear path toward correcting the underlying model because “changing one thing changes everything” in a neural network (Sculley et al. 2015). Our article focuses on addressing the limitations of these local and post-hoc ex- plainability approaches by …

Content truncated to 500 words