Towards Stable Natural Language Understanding via Information Entropy Guided Debiasing

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2868–2882 July 9-14, 2023 ©2023 Association for Computational Linguistics Towards Stable Natural Language Understanding via Information Entropy Guided Debiasing Li Du †1,2 , Xiao Ding ∗†1 , Zhouhao Sun1, Ting Liu1 , Bing Qin1 , and Jingshuo Liu1 1 Research Center for Social Computing and Information Retrieval Harbin Institute of Technology, China 2 Beijing Academy of Artificial Intelligence, Beijing, China {ldu, xding, hzsun, tliu, qinb}@ir.hit.edu.cn jingshuoliu@stu.hit.edu.cn Abstract Although achieving promising performance, current Natural Language Understanding mod- els tend to utilize dataset biases instead of learn- ing the intended task, which always leads to performance degradation on out-of-distribution (OOD) samples. To increase the performance stability, previous debiasing methods empiri- cally capture bias features from data to prevent the model from corresponding biases. However, our analyses show that the empirical debiasing methods may fail to capture part of the dataset biases and mistake semantic information of in- put text as biases, which limits the effective- ness of debiasing. To address these issues, we propose a debiasing framework IEGDB that comprehensively detects the dataset biases to induce a set of biased features, and purify the biased features with the guidance of informa- tion entropy. Experimental results show that IEGDB can consistently improve the stability of performance on OOD datasets for a set of widely adopted NLU models. 1 Introduction The Natural Language Understanding (NLU) task requires a model to understand the semantics of input text and then infer the target label. State-of- the-Art NLU models such as BERT have achieved impressive performance on various NLU tasks (De- vlin et al., 2019; Liu et al., 2019). However, recent analyses have demonstrated that these models may exploit the dataset biases, i.e., superficial surface cues that are spuriously associated with the target labels for making inferences (McCoy et al., 2019; Zellers et al., 2019; Utama et al., 2020a). This leads to performance degradation on out-of-distribution (OOD) challenge sets that are designed for making models relying on spurious associations obtaining incorrect predictions (McCoy et al., 2019; Zhang et al., 2019; He et al., 2019). *Corresponding Author †These authors contributed equally to this work To increase the stability of model performance on OOD samples, debiasing methods are proposed to mitigate the influence of dataset biases. In gen- eral, the debiasing methods work by first extracting a set of biased features characterizing the dataset bi- ases, then regularizing the main NLU model using the biased features by various existing regularizers, to prevent it from fitting dataset biases (Schuster et al., 2019; Clark et al., 2019; Utama et al., 2020a). Hence, the key of debiasing lies in how to identify the dataset bias and extract corresponding biased features. Early debiasing methods rely on the prior knowl- edge of researchers to design biased features (He et al., 2019; Clark et al., 2019; Mahabadi et al., 2020). However, the assumption that the types of biases should be known a-priori limits their appli- cation to many …

Content truncated to 500 words