MetaSLRCL: A Self-Adaptive Learning Rate and Curriculum Learning Based Framework for Few-Shot Text Classification

Proceedings of the 29th International Conference on Computational Linguistics, pages 2065–2074 October 12–17, 2022. 2065 MetaSLRCL: A Self-Adaptive Learning Rate and Curriculum Learning Based Framework for Few-Shot Text Classification Kailin Zhao, Xiaolong Jin∗, Saiping Guan, Jiafeng Guo, Xueqi Cheng CAS Key Laboratory of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences; School of Computer Science and Technology, University of Chinese Academy of Sciences {zhaokailin17z, jinxiaolong, guansaiping}@ict.ac.cn {guojiafeng, cxq}@ict.ac.cn Abstract Due to the lack of labeled data in many real- istic scenarios, a number of few-shot learning methods for text classification have been pro- posed, among which the meta learning based ones have recently attracted much attention. Such methods usually consist of a learner as the classifier and a meta learner for special- izing the learner to different tasks. For the learner, learning rate is crucial to its perfor- mance. However, existing methods treat it as a hyper parameter and adjust it manually, which is time-consuming and laborious. Intuitively, for different tasks and neural network layers, the learning rates should be different and self- adaptive. For the meta learner, it requires a good generalization ability so as to quickly adapt to new tasks. Motivated by these issues, we propose a novel meta learning framework, called MetaSLRCL, for few-shot text classifi- cation. Specifically, we present a novel meta learning mechanism to obtain different learning rates for different tasks and neural network lay- ers so as to enable the learner to quickly adapt to new training data. Moreover, we propose a task-oriented curriculum learning mechanism to help the meta learner achieve a better gener- alization ability by learning from different tasks with increasing difficulties. Extensive experi- ments on three benchmark datasets demonstrate the effectiveness of MetaSLRCL. 1 Introduction Text classification is one of the most concerned tasks in Natural Language Processing (NLP). At present, most text classification methods are based on supervised learning with a large amount of la- beled data. But there is not so much labeled data, even source data, in many scenarios (e.g., news classification in specific domains). Some distant supervision methods (Mintz et al., 2009) have thus been proposed to handle this problem. However, ∗*Corresponding author. this kind of methods may add a large proportion of noisy training data (Zeng et al., 2014). Because of this, it is a big challenge for traditional supervised learning methods to work well in the scenarios with very limited training data. As a result, few-shot text classification has attracted much attention in recent years, where there are only a few labeled instances available for each class. The concept of few-shot learning was formally put forward by (Li et al., 2003). They presented a method for learning from classes with few data, by incorporating generic knowledge which may be obtained from previously learned models of unrelated classes. The existing few-shot learning methods can be divided into three categories (Gao et al., 2019), namely, model fine-tuning based (e.g., (Howard and Ruder, 2018; Nakamura and Harada, 2019)), metric learning based (e.g., …

Content truncated to 500 words