Types of Out-of-Distribution Texts and How to Detect Them

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10687–10701 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 10687 Types of Out-of-Distribution Texts and How to Detect Them Udit Arora♠ William Huang♣∗ He He♠ ♠New York University ♣Capital One {uditarora,hhe}@nyu.edu, william.huang@capitalone.com Abstract Despite agreement on the importance of detect- ing out-of-distribution (OOD) examples, there is little consensus on the formal definition of OOD examples and how to best detect them. We categorize these examples by whether they exhibit a background shift or a semantic shift, and find that the two major approaches to OOD detection, model calibration and density estimation (language modeling for text), have distinct behavior on these types of OOD data. Across 14 pairs of in-distribution and OOD En- glish natural language understanding datasets, we find that density estimation methods consis- tently beat calibration methods in background shift settings, while performing worse in se- mantic shift settings. In addition, we find that both methods generally fail to detect examples from challenge data, highlighting a weak spot for current methods. Since no single method works well across all settings, our results call for an explicit definition of OOD examples when evaluating different detection methods. 1 Introduction Current NLP models work well when the training and test distributions are the same (e.g. from the same benchmark dataset). However, it is common to encounter out-of-distribution (OOD) examples that diverge from the training data once the model is deployed to real settings. When training and test distributions differ, current models tend to pro- duce unreliable or even catastrophic predictions that hurt user trust (Ribeiro et al., 2020). Therefore, it is important to identify OOD inputs so that we can modify models’ inference-time behavior by ab- staining, asking for human feedback, or gathering additional information (Amodei et al., 2016). Current work in NLP either focuses on spe- cific tasks like intent classification in task- oriented dialogue (Zheng et al., 2020), or arbi- trary in-distribution (ID) and OOD dataset pairs ∗Work done while at New York University. semantic background density estimation score: p(x) semantic background failure case calibration score: p(y x) Darker color indicates higher likelihood of OOD ID positive ID negative OOD Figure 1: Illustration of semantic shift and background shift in R2. Each point consists of semantic features (x-axis) and background features (y-axis). OOD exam- ples (red points) can shift in either direction. The back- ground color indicates regions of ID (light) and OOD (dark) given by the density estimation method (left) and the calibration method (right). The calibration method fails to detect OOD examples due to background shift. (Hendrycks et al., 2020b, 2019; Zhou and Chen, 2021), e.g. taking a sentiment classification dataset as ID and a natural language inference dataset as OOD. However, getting inputs intended for a different task is rare in realistic settings as users typically know the intended task. In practice, an example is considered OOD due to various rea- sons, e.g. being rare (Sagawa et al., 2020), out-of- domain (Daumé III, 2007), …

Content truncated to 500 words