Prompting Contrastive Explanations for Commonsense Reasoning Tasks

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4179–4192 August 1–6, 2021. ©2021 Association for Computational Linguistics 4179 Prompting Contrastive Explanations for Commonsense Reasoning Tasks Bhargavi Paranjape†∗ Julian Michael† Marjan Ghazvininejad∗ Hannaneh Hajishirzi†ϵ Luke Zettlemoyer†∗ †Allen School of Computer Science & Engineering, University of Washington, Seattle, WA ϵAllen Institute of Artificial Intelligence, Seattle ∗Facebook AI {bparan,julianjm,lsz,hannaneh}@cs.washington.edu Abstract Many commonsense reasoning NLP tasks in- volve choosing between one or more possi- ble answers to a question or prompt based on knowledge that is often implicit. Large pre- trained language models (PLMs) can achieve near-human performance on such tasks, while providing little human-interpretable evidence of the underlying reasoning they use. In this work, we show how to use these same models to generate such evidence: inspired by the contrastive nature of human explana- tions, we use PLMs to complete explanation prompts which contrast alternatives according to the key attribute(s) required to justify the correct answer (for example, peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts peanuts are usu- ally salty while raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins raisins are sweet). Condition- ing model decisions on these explanations im- proves performance on two commonsense rea- soning benchmarks, as compared to previous non-contrastive alternatives. These explana- tions are also judged by humans to be more relevant for solving the task, and facilitate a novel method to evaluate explanation faithful- ness. 1 Introduction Pretrained Language Models (PLMs) (Raffel et al., 2020; Lewis et al., 2020; Radford et al., 2019; Brown et al., 2020) have been shown to encode sub- stantial amounts of knowledge in their parameters (Petroni et al., 2019; Talmor et al., 2020; Roberts et al., 2020) and have achieved impressive perfor- mance on commonsense reasoning (CSR) tasks without the use of external knowledge (Trinh and Le, 2018; Yang et al., 2020). However, these mod- els provide little human-interpretable evidence of the intermediate commonsense knowledge or rea- soning they use, and have been observed to overly rely on superficial dataset artifacts (Poliak et al., 2018; Geva et al., 2019). To overcome this lim- itation, recent work has shown that PLMs can i) I picked up a bag of peanuts and raisins for a snack. I wanted a sweeter snack out so I ate the for now. Contrastive Expl. - Peanuts are salty while raisins tend to be sweet. ii) The geese prefer to nest in the fields rather than the forests because in the predators are more hidden. Contrastive Expl. - Forests are denser than fields Table 1: Examples of Winograd Schema Instances where the correct and incorrect answer choices are highlighted in blue and red respectively. Choices are contrasted along attributes like taste (for i) and density of vegetation (for ii) by humans to explain why they prefer some answer choice. explain themselves by generating free-form nat- ural language explanations of their reasoning pat- terns (Rajani et al., 2019a; Camburu et al., 2018; …

Content truncated to 500 words