Extending Logic Explained Networks to Text Classification

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8838 - 8857 December 7-11, 2022 ©2022 Association for Computational Linguistics Extending Logic Explained Networks to Text Classification Rishabh Jain∗ University of Cambridge Cambridge, UK Gabriele Ciravegna Université Côte d’Azur, Inria, CNRS, I3S, Maasai, Nice, France Pietro Barbiero University of Cambridge Cambridge, UK Francesco Giannini University of Siena Siena, Italy Davide Buffelli University of Cambridge Cambridge, UK Pietro Lio University of Cambridge Cambridge, UK Abstract Recently, Logic Explained Networks (LENs) have been proposed as explainable-by-design neural models providing logic explanations for their predictions. However, these models have only been applied to vision and tabular data, and they mostly favour the generation of global explanations, while local ones tend to be noisy and verbose. For these reasons, we propose LENp, improving local explanations by perturb- ing input words, and we test it on text classifi- cation. Our results show that (i) LENp provides better local explanations than LIME in terms of sensitivity and faithfulness, and (ii) logic expla- nations are more useful and user-friendly than feature scoring provided by LIME as attested by a human survey. 1 Introduction The development of Deep Neural Networks has enabled the creation of high accuracy text clas- sifiers (LeCun et al., 2015) with state-of-the-art models leveraging different forms of architectures, like RNNs (GRU, LSTM) (Minaee et al., 2020) or Transformer models (Vaswani et al., 2017). How- ever, these architectures are considered as black- box models (Adadi and Berrada, 2018), since their decision processes are not easy to explain and de- pend on a very large set of parameters. In or- der to shed light on neural models’ decision pro- cesses, eXplainable Artificial Intelligence (XAI) techniques attempt to understand text attribution to certain classes, for instance by using white-box models. Interpretable-by-design models engender higher trust in human users with respect to explana- tion methods for black-boxes, at the cost, however, of lower prediction performance. Recently, Ciravegna et al. (2021) and Barbi- ero et al. (2022) introduced the Logic Explained Network (LEN), an explainable-by-design neural network combining interpretability of white-box ∗ Corresponding author: rj412@cam.ac.uk models with high performance of neural networks. However, the authors only compared LENs with white-box models and on tabular/computer vision tasks. For this reason, in this work we apply an im- proved version of the LEN (LENp) to the text clas- sification problem, and we compare it with LIME (Ribeiro et al., 2016), a very-well known expla- nation method. LEN and LIME provide different kind of explanations, respectively FOL formulae and feature-importance vectors, and we assess their user-friendliness by means of a user-study. As an evaluation benchmark, we considered Multi-Label Text Classification for the tag classification task on the “StackSample: 10% of Stack Overflow Q&A” dataset (Overflow, 2019). Contribution The paper aims to: (i) improve LEN explanation algorithm (LENp)1; (ii) compare the faithfulness and the sensitivity of the explana- tions provided by LENs and LIME; (iii) assess the user-friendliness of the two kinds of explanations. 2 Background Explainable AI Explainable …

Content truncated to 500 words