Counterfactual Active Learning for Out-of-Distribution Generalization

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 11362–11377 July 9-14, 2023 ©2023 Association for Computational Linguistics Counterfactual Active Learning for Out-of-Distribution Generalization Xun Deng 1, Wenjie Wang 3∗, Fuli Feng 1,2, Hanwang Zhang 4, Xiangnan He 1, Yong Liao 5, 1 University of Science and Technology of China, Hefei 230026, China 2 Institute of Dataspace, Hefei, Anhui, China 3 National University of Singapore 4 Nanyang Technological University 5 China Academic of Electronics and Information Technology {dx981228,wenjiewang96,fulifeng93,xiangnanhe}@gmail.com hanwangzhang@ntu.edu.sg, yliao@ustc.edu.cn Abstract We study the out-of-distribution generalization of active learning that adaptively selects sam- ples for annotation in learning the decision boundary of classification. Our empirical study finds that increasingly annotating seen samples may hardly benefit the generalization. To ad- dress the problem, we propose Counterfactual Active Learning (CounterAL) that empowers active learning with counterfactual thinking to bridge the seen samples with unseen cases. In addition to annotating factual samples, Coun- terAL requires annotators to answer counterfac- tual questions to construct counterfactual sam- ples for training. To achieve CounterAL, we design a new acquisition strategy that selects the informative factual-counterfactual pairs for annotation; and a new training strategy that pushes the model update to focus on the dis- crepancy between factual and counterfactual samples. We evaluate CounterAL on multi- ple public datasets of sentiment analysis and natural language inference. The experiment results show that CounterAL requires fewer acquisition rounds and outperforms existing active learning methods by a large margin in OOD tests with comparable IID performance. 1 Introduction Active Learning (AL) is widely applied to allevi- ate the scarcity of labeled data in various machine learning applications (Ren et al., 2021) such as fi- nancial fraud detection (Carcillo et al., 2017) where the annotation cost is high. Existing research on AL mainly focuses on the design of an acquisition strat- egy that adaptively selects informative samples for annotation from an unlabeled pool (Tan et al., 2021; Kirsch et al., 2019). While the models learned by AL methods have comparable in-distribution per- formance with the ones learned from fully labeled ∗ Wenjie Wang is the corresponding author. This work is supported by the National Key Research and Development Program of China (2021YFC3300500-02), the National Natu- ral Science Foundation of China (62272437 and U20B2047), and the CCCD Key Lab of Ministry of Culture and Tourism. 0.3 0.4 0.5 0.6 0.7 0.8 1 2 3 4 5 6 7 8 9 10 Accuracy Round IID OOD (a) Natural Language Inference 0.5 0.6 0.7 0.8 0.9 1 1 2 3 4 5 6 7 8 9 10 Accuracy Round IID OOD (b) Sentiment Analysis Figure 1: The performance on IID and OOD tests of sentiment analysis and natural language inference datasets (Kaushik et al., 2020) in the learning procedure of the AL method: Entropy. data, they typically result in poor generalization on out-of-distribution samples (Krishnan et al., 2021). As OOD samples widely exist in practice (Wang et al., 2022), it is critical to enhance the OOD gen- …

Content truncated to 500 words