Fine-Tuning Deteriorates General Textual Out-of-Distribution Detection by Distorting Task-Agnostic Features

Findings of the Association for Computational Linguistics: EACL 2023, pages 564–579 May 2-6, 2023 ©2023 Association for Computational Linguistics Fine-Tuning Deteriorates General Textual Out-of-Distribution Detection by Distorting Task-Agnostic Features Sishuo Chen1, Wenkai Yang1, Xiaohan Bi1, Xu Sun2 1Center for Data Science, Peking University 2MOE Key Laboratory of Computational Linguistics, School of Computer Science, Peking University {chensishuo,xusun}@pku.edu.cn {wkyang,bxh}@stu.pku.edu.cn Abstract Detecting out-of-distribution (OOD) inputs is crucial for the safe deployment of natural lan- guage processing (NLP) models. Though ex- isting methods, especially those based on the statistics in the feature space of fine-tuned pre- trained language models (PLMs), are claimed to be effective, their effectiveness on different types of distribution shifts remains underex- plored. In this work, we take the first step to comprehensively evaluate the mainstream textual OOD detection methods for detecting semantic and non-semantic shifts. We find that: (1) no existing method behaves well in both settings; (2) fine-tuning PLMs on in- distribution data benefits detecting semantic shifts but severely deteriorates detecting non- semantic shifts, which can be attributed to the distortion of task-agnostic features. To alle- viate the issue, we present a simple yet ef- fective general OOD score named GNOME that integrates the confidence scores derived from the task-agnostic and task-specific repre- sentations. Experiments show that GNOME works well in both semantic and non-semantic shift scenarios, and further brings significant improvement on two cross-task benchmarks where both kinds of shifts simultaneously take place. Our code is available at https:// github.com/lancopku/GNOME. 1 Introduction The pre-training and fine-tuning paradigm based on Transformers (Vaswani et al., 2017) has achieved tremendous success in various natural language understanding (NLU) tasks (Devlin et al., 2019; Liu et al., 2019; Qiu et al., 2020). However, fine-tuned pre-trained language models (PLMs) notoriously suffer from over-confident predictions on out-of- distribution (OOD) inputs (Hendrycks et al., 2020). As this issue threats the reliability of NLP models deployed in the open world, textual OOD detection has attracted great attention recently (Podolskiy et al., 2021; Zhou et al., 2021, 2022; Duan et al., Figure 1: OOD detection performance (FAR95↓, lower is better) in non-semantic and semantic shift scenarios. No single existing method works well in both scenarios, but our proposed GNOME mitigates the trade-off. 2022, etc.), which aims to enable the model to abstain from making unreasonable predictions on OOD data and resort to human intervention. Nonetheless, almost all of the current approaches are assessed under certain assumptions about the type of OOD texts. One line of works creates in-distribution (ID) and OOD pairs from arbitrary datasets for different tasks (Hendrycks et al., 2020), while another line assumes that OOD data belong to classes in the ID task but unseen during train- ing, e.g., in intent recognition (Podolskiy et al., 2021). Arora et al. (2021) reveal the inconsistency among the evaluation protocols and category the distribution shifts to non-semantic shifts (NSS) and semantic shifts (SS), but a thorough comparison of existing methods in different settings is miss- ing as later works either focus on either detecting NSS (Duan et al., …

Content truncated to 500 words