FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 8646 - 8665 May 22-27, 2022 c⃝2022 Association for Computational Linguistics FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning Jing Zhou ∗1 Yanan Zheng ∗24 Jie Tang 24 Jian Li †1 Zhilin Yang †13 1Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University 2Department of Computer Science and Technology, Tsinghua University 3Shanghai Qi Zhi Institute 4Beijing Academy of Artificial Intelligence (BAAI) Abstract Most previous methods for text data augmenta- tion are limited to simple tasks and weak base- lines. We explore data augmentation on hard tasks (i.e., few-shot natural language under- standing) and strong baselines (i.e., pretrained models with over one billion parameters). Un- der this setting, we reproduced a large number of previous augmentation methods and found that these methods bring marginal gains at best and sometimes degrade the performance much. To address this challenge, we propose a novel data augmentation method FlipDA that jointly uses a generative model and a classifier to gen- erate label-flipped data. Central to the idea of FlipDA is the discovery that generating label- flipped data is more crucial to the performance than generating label-preserved data. Experi- ments show that FlipDA achieves a good trade- off between effectiveness and robustness—it substantially improves many tasks while not negatively affecting the others.1 1 Introduction Data augmentation is a method to augment the training set by generating new data from the given data. For text data, basic operations including re- placement, insertion, deletion, and shuffle have been adopted widely and integrated into a wide range of augmentation frameworks (Zhang et al., 2015; Wang and Yang, 2015; Xie et al., 2020a; Kobayashi, 2018; Wei and Zou, 2019). Generative modeling methods such as back-translation have also been employed to generate augmented samples (Fadaee et al., 2017; Sennrich et al., 2016). How- ever, there are two major limitations. First, some general augmentation methods are based on weak baselines without using large-scale pretrained lan- guage models. Recent work showed that some of ∗The authors have contributed equally to this work. † Corresponding Authors. 1Code is available at https://github.com/ zhouj8553/FlipDA the data augmentation methods are less useful when combined with large pretrained models (Longpre et al., 2020). Second, most prior studies are carried on simple tasks such as single-sentence classifica- tion where it is easier to generate legit augmented samples. For harder tasks such as natural language inference (e.g., telling whether sentence A entails sentence B), it is not clear whether previous meth- ods still help. This work takes a step further to study data aug- mentation under strong baselines and hard tasks. Our study employs large-scale pretrained language models such as DeBERTa (He et al., 2020c) with over one billion parameters as baselines. More- over, we target a very challenging setting—few- shot natural language understanding (NLU). Fol- lowing (Schick and Schutze, 2021), we consider challenging NLU tasks including question answer- ing, textual entailment, coreference resolution, and word sense disambiguation, and use only 32 train- …

Content truncated to 500 words