Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428–3448 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3428 Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference R. Thomas McCoy,1 Ellie Pavlick,2 & Tal Linzen1 1Department of Cognitive Science, Johns Hopkins University 2Department of Computer Science, Brown University tom.mccoy@jhu.edu, ellie pavlick@brown.edu, tal.linzen@jhu.edu Abstract A machine learning system can score well on a given test set by relying on heuristics that are effective for frequent example types but break down in more challenging cases. We study this issue within natural language inference (NLI), the task of determining whether one sentence entails another. We hypothesize that statisti- cal NLI models may adopt three fallible syn- tactic heuristics: the lexical overlap heuristic, the subsequence heuristic, and the constituent heuristic. To determine whether models have adopted these heuristics, we introduce a con- trolled evaluation set called HANS (Heuris- tic Analysis for NLI Systems), which contains many examples where the heuristics fail. We find that models trained on MNLI, including BERT, a state-of-the-art model, perform very poorly on HANS, suggesting that they have indeed adopted these heuristics. We conclude that there is substantial room for improvement in NLI systems, and that the HANS dataset can motivate and measure progress in this area. 1 Introduction Neural networks excel at learning the statistical patterns in a training set and applying them to test cases drawn from the same distribution as the training examples. This strength can also be a weakness: statistical learners such as standard neural network architectures are prone to adopting shallow heuristics that succeed for the majority of training examples, instead of learning the underly- ing generalizations that they are intended to cap- ture. If such heuristics often yield correct outputs, the loss function provides little incentive for the model to learn to generalize to more challenging cases as a human performing the task would. This issue has been documented across domains in artificial intelligence. In computer vision, for example, neural networks trained to recognize ob- jects are misled by contextual heuristics: a net- work that is able to recognize monkeys in a typ- ical context with high accuracy may nevertheless label a monkey holding a guitar as a human, since in the training set guitars tend to co-occur with hu- mans but not monkeys (Wang et al., 2018). Sim- ilar heuristics arise in visual question answering systems (Agrawal et al., 2016). The current paper addresses this issue in the do- main of natural language inference (NLI), the task of determining whether a premise sentence entails (i.e., implies the truth of) a hypothesis sentence (Condoravdi et al., 2003; Dagan et al., 2006; Bow- man et al., 2015). As in other domains, neural NLI models have been shown to learn shallow heuris- tics, in this case based on the presence of specific words (Naik et al., 2018; Sanchez et al., 2018). For example, a model might assign a label of contra- diction to …

Content truncated to 500 words