Looking at the Overlooked: An Analysis on the Word-Overlap Bias in Natural Language Inference

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10605 - 10616 December 7-11, 2022 ©2022 Association for Computational Linguistics Looking at the Overlooked: An Analysis on the Word-Overlap Bias in Natural Language Inference Sara Rajaee1, Yadollah Yaghoobzadeh2, and Mohammad Taher Pilehvar3 1 University of Amsterdam, Netherlands 2 University of Tehran, Iran 3 Tehran Institute for Advanced Studies, Khatam University, Iran s.rajaee@uva.nl y.yaghoobzadeh@ut.ac.ir mp792@cam.ac.uk Abstract It has been shown that NLI models are usu- ally biased with respect to the word-overlap between premise and hypothesis; they take this feature as a primary cue for predicting the entailment label. In this paper, we fo- cus on an overlooked aspect of the overlap bias in NLI models: the reverse word-overlap bias. Our experimental results demonstrate that current NLI models are highly biased to- wards the non-entailment label on instances with low overlap, and the existing debiasing methods, which are reportedly successful on existing challenge datasets, are generally inef- fective in addressing this category of bias. We investigate the reasons for the emergence of the overlap bias and the role of minority ex- amples in its mitigation. For the former, we find that the word-overlap bias does not stem from pre-training, and for the latter, we ob- serve that in contrast to the accepted assump- tion, eliminating minority examples does not affect the generalizability of debiasing meth- ods with respect to the overlap bias. All the code and relevant data are available at: https: //github.com/sara-rajaee/reverse_bias 1 Introduction Natural Language Inference (NLI) is one of the most commonly used NLP tasks, particularly in the scope of evaluating models for their language understanding capabilities. Since their emergence, pre-trained language models (PLMs) have been highly successful on standard NLI datasets, such as the Multi-Genre Natural Language Inference (Williams et al., 2018, MultiNLI). However, recent analytical studies have revealed that their success is partly due to their reliance on spurious correlations between superficial features of the input texts and gold labels in these datasets (Poliak et al., 2018; Bhargava et al., 2021). As a result, performance usually drops on out-of-distribution datasets where such correlations do not hold. Several proposals Figure 1: NLI model’s confidence on a randomly sam- pled subset of instances from the SNLI dataset across four different degrees of word overlap between premise and hypothesis. BERT is biased towards the entail- ment label on instances with full overlap (denoted by the huge confidence gap with the non-entailment label). On the contrary, a reverse bias is seen for low and non- overlapping instances, with a significant confidence lead on the non-entailment label. have been put forth to enhance the robustness of models to the known and unknown biases and im- prove performance on the so-called challenging datasets (Stacey et al., 2020; Utama et al., 2020a; Asael et al., 2022). One of the well-known dataset biases in NLI models is the spurious correlation of the entail- ment label and high word-overlap between premise and hypothesis. A number of challenging sets are designed to …

Content truncated to 500 words