Adaptive Meta-learner via Gradient Similarity for Few-shot Text Classification

Proceedings of the 29th International Conference on Computational Linguistics, pages 4873–4882 October 12–17, 2022. 4873 Adaptive Meta-learner via Gradient Similarity for Few-shot Text Classification Tianyi Lei1, Honghui Hu1, Qiaoyang Luo2, Dezhong Peng1, Xu Wang1∗ 1College of Computer Science, Sichuan University 2The University of Adelaide {leity828,wangxu.scu}@gmail.com Abstract Few-shot text classification aims to classify the text under the few-shot scenario. Most of the previous methods adopt optimization-based meta learning to obtain task distribution. How- ever, due to the neglect of matching between the few amount of samples and complicated models, as well as the distinction between use- ful and useless task features, these methods suf- fer from the overfitting issue. To address this is- sue, we propose a novel Adaptive Meta-learner via Gradient Similarity (AMGS) method to im- prove the model generalization ability to a new task. Specifically, the proposed AMGS allevi- ates the overfitting based on two aspects: (i) acquiring the potential semantic representation of samples and improving model generalization through the self-supervised auxiliary task in the inner loop, (ii) leveraging the adaptive meta- learner via gradient similarity to add constraints on the gradient obtained by base-learner in the outer loop. Moreover, we make a systematic analysis of the influence of regularization on the entire framework. Experimental results on several benchmarks demonstrate that the pro- posed AMGS consistently improves few-shot text classification performance compared with the state-of-the-art optimization-based meta- learning approaches. The code is available at: https://github.com/Tianyi-Lei. 1 Introduction As a fundamental task of few-shot learning (Fei- Fei et al., 2006) in natural language processing theme, few-shot text classification (Yu et al., 2018; Geng et al., 2019) requires a model to predict cat- egories that are not seen in training. Meta learn- ing (Schmidhuber, 1987; Thrun and Pratt, 2012), which plays a crucial role in general few-shot learn- ing, aims to improve generalization ability and fast adaptation ability of the learner through modelling the distribution of tasks. To adapt few-shot tasks, ∗Corresponding author typical supervised meta-learning methods (Vinyals et al., 2016; Finn et al., 2017) model task distribu- tions from a few support tasks over meta-training episodes. Subsequently, numerous methods based on meta-learning (Bao et al., 2020; Luo et al., 2021; Han et al., 2021) are proposed to solve few-shot text classification problem. Within the meta-learning frameworks, Bao et al. (2020) trains an attention-based model to enhance the text representation of distributional signature, Luo et al. (2021) leverages label-semantic augmen- tation to help BERT compensate for the ambiguity of the class definition caused by the limited data, and Han et al. (2021) strengthens the generalization of a model using an adversarial domain adaptation network. However, these methods are similar to the traditional meta-learning methods, neglecting the overfitting problem caused by utilizing the few number of data in the complicated models under the meta-learning frameworks. To address the above problem in few-shot text classification, several methods are proposed based on a principle i.e., obtaining more task-distribution can ameliorate the risk of over-fitting to the train- ing task distribution. Bansal et al. (2020) …

Content truncated to 500 words