Learning Explainable Linguistic Expressions with Neural Inductive Logic Programming for Sentence Classification

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 4211–4221, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 4211 Learning Explainable Linguistic Expressions with Neural Inductive Logic Programming for Sentence Classification Prithviraj Sen IBM Research San Jose, CA, USA senp@us.ibm.com Marina Danilevsky IBM Research San Jose, CA, USA mdanile@us.ibm.com Yunyao Li IBM Research San Jose, CA, USA yunyaoli@us.ibm.com Siddhartha Brahma Google Research Mountain View, CA, USA sidbrahma@google.com Matthias Boehm Graz University of Technology Graz, Austria m.boehm@tugraz.at Laura Chiticariu IBM Watson San Jose, CA, USA chiti@us.ibm.com Rajasekar Krishnamurthy IBM Watson San Jose, CA, USA rajase@us.ibm.com Abstract Interpretability of predictive models is becom- ing increasingly important with growing adop- tion in the real-world. We present RuleNN, a neural network architecture for learning trans- parent models for sentence classification. The models are in the form of rules expressed in first-order logic, a dialect with well-defined, human-understandable semantics. More pre- cisely, RuleNN learns linguistic expressions (LE) built on top of predicates extracted us- ing shallow natural language understanding. Our experimental results show that RuleNN outperforms statistical relational learning and other neuro-symbolic methods, and performs comparably with black-box recurrent neural networks. Our user studies confirm that the learned LEs are explainable and capture do- main semantics. Moreover, allowing domain experts to modify LEs and instill more do- main knowledge leads to human-machine co- creation of models with better performance. 1 Introduction Difficult-to-interpret, black-box predictive models have been shown to harbor undesirable biases (e.g., racial bias in computing risk of recidivism among criminals (Angwin et al., 2016; Liptak, 2017)). Re- newed interest in interpretability (BlackBoxNLP) has led to techniques for explaining not only the in- ner workings of the model but also to explain how it derives a prediction. While various techniques for explainability exist (see survey by Guidotti et al. (2018)), one popular approach explains predictions from a black-box model by using a surrogate models (Ribeiro et al., 2016). Another extracts explanations from neural network layer activations, especially when said activations appeal to human intuition such as at- tention (Bahdanau et al., 2015) which may be interpreted as importance weights assigned to (la- tent) features derived by the model. While such approaches are useful, they raise questions such as whether the purported explanation provided by the surrogate correctly reflects the process employed by the black-box model to arrive at the prediction (sometimes called inexact explanation (Chu et al., 2018)). Similarly, attention only provides noisy explanations (Serrano and Smith, 2019). Such ap- proaches leave room for improvement because ex- plainability is treated as an after-thought whereas our goal is to treat it as a first-class citizen. In other words, is it possible to devise a neural network that directly learns a model expressed in a clear, human-readable dialect? First-order logic (FOL) is a human-interpretable fragment of logic that includes existential (∃) and universal (∀) quantification along with proposi- tional logic’s conjunction (∧), disjunction (∨) and negation (¬) operators. FOL can model diverse applications such as data integration (Singla and Domingos, 2006), image …

Content truncated to 500 words