Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4157–4165 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 4157 Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations Oana-Maria Camburu1,4 Brendan Shillingford1,2 Pasquale Minervini3 Thomas Lukasiewicz1,4 Phil Blunsom1,2 1 University of Oxford 2 DeepMind, London 3 University College London 4 Alan Turing Institute, London firstname.lastname@cs.ox.ac.uk p.minervini@ucl.ac.uk Abstract To increase trust in artificial intelligence sys- tems, a promising research direction consists of designing neural models capable of generat- ing natural language explanations for their pre- dictions. In this work, we show that such mod- els are nonetheless prone to generating mu- tually inconsistent explanations, such as “Be- cause there is a dog in the image.” and “Be- cause there is no dog in the [same] image.”, exposing flaws in either the decision-making process of the model or in the generation of the explanations. We introduce a simple yet ef- fective adversarial framework for sanity check- ing models against the generation of incon- sistent natural language explanations. More- over, as part of the framework, we address the problem of adversarial attacks with full target sequences, a scenario that was not pre- viously addressed in sequence-to-sequence at- tacks. Finally, we apply our framework on a state-of-the-art neural natural language infer- ence model that provides natural language ex- planations for its predictions. Our framework shows that this model is capable of generating a significant number of inconsistent explana- tions. 1 Introduction In order to explain the predictions produced by accurate yet black-box neural models, a growing number of works propose extending these models with natural language explanation generation mod- ules, thus obtaining models that explain themselves in human language (Hendricks et al., 2016; Cam- buru et al., 2018; Park et al., 2018; Kim et al., 2018; Ling et al., 2017). In this work, we first draw attention to the fact that such models, while appealing, are nonethe- less prone to generating inconsistent explanations. We define two explanations to be inconsistent if they provide contradictory arguments about the in- stances and predictions that they aim to explain. For example, consider a visual question answering (VQA) task (Park et al., 2018) and two instances where the image is the same but the questions are different, say “Is there an animal in the image?” and “Can you see a Husky in the image?”. If for the first instance a model predicts “Yes.” and gen- erates the explanation “Because there is a dog in the image.”, while for the second instance the same model predicts “No.” and generates the explanation “Because there is no dog in the image.”, then the model is producing inconsistent explanations. Inconsistent explanations reveal at least one of the following undesired behaviors: (i) at least one of the explanations is not faithfully describing the decision mechanism of the model, or (ii) the model relied on a faulty decision mechanism for at least one of the instances. Note that, for a pair of incon- sistent explanations, further …

Content truncated to 500 words