Effective Few-Shot Classification with Transfer Learning

Proceedings of the 28th International Conference on Computational Linguistics, pages 1061–1066 Barcelona, Spain (Online), December 8-13, 2020 1061 Effective Few-Shot Classification with Transfer Learning Aakriti Gupta Yahoo! Research Kapil Thadani Yahoo! Research {aakriti.gupta,thadani,nohare}@verizonmedia.com Neil O’Hare Yahoo! Research Abstract Few-shot learning addresses the the problem of learning based on a small amount of training data. Although more well-studied in the domain of computer vision, recent work has adapted the Amazon Review Sentiment Classification (ARSC) text dataset for use in the few-shot setting. In this work, we use the ARSC dataset to study a simple application of transfer learning approaches to few-shot classification. We train a single binary classifier to learn all few-shot classes jointly by prefixing class identifiers to the input text. Given the text and class, the model then makes a binary prediction for that text/class pair. Our results show that this simple approach can outperform most published results on this dataset. Surprisingly, we also show that including domain information as part of the task definition only leads to a modest improvement in model accuracy, and zero- shot classification, without further fine-tuning on few-shot domains, performs equivalently to few-shot classification. These results suggest that the classes in the ARSC few-shot task, which are defined by the intersection of domain and rating, are actually very similar to each other, and that a more suitable dataset is needed for the study of few-shot text classification. 1 Introduction To deal with emerging topics, events or linguistic phenomena, text classification systems often need to accommodate new classes. Recently, there has been increasing interest in addressing this problem through the lens of few-shot learning, which focuses on learning from a small handful of examples rather than a large annotated dataset. Few-shot classification research in computer vision and natural language processing studies the problem of generalizing from classes with plentiful training data to previously unseen classes for which labeled data may not be widely available. At the same time, there has been a widespread growth of interest in transfer learning for natural lan- guage processing, with transformer-based self-supervised models such as BERT (Devlin et al., 2019) and T5 (Raffel et al., 2019) achieving strong results on a variety of benchmark problems. These models can be fine-tuned to new tasks with a small number of training examples, suggesting that their generalization capabilities may also be applicable to the few-shot learning setting. In this paper, we study a straightforward transfer learning approach applied to a standard few-shot text classification dataset. Specifically, we focus on the well-studied Amazon Review Sentiment Classi- fication (ARSC) dataset, which was originally developed for domain adaptation research (Blitzer et al., 2007), and was adapted to the few-shot setting by Yu et al. (2018). Various few-shot learning approaches have been benchmarked on this dataset, including techniques based on metric learning (Vinyals et al., 2016) and meta-learning (Finn et al., 2017). In this work, we consider a simple BERT-based classification scheme that first fine-tunes a pretrained model on the full rating classification dataset, and then further …

Content truncated to 500 words