Variance-reduced First-order Meta-learning for Natural Language Processing Tasks

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2609–2615 June 6–11, 2021. ©2021 Association for Computational Linguistics 2609 Variance-reduced First-order Meta-learning for Natural Language Processing Tasks Lingxiao Wang1, Kevin Huang2, Tengyu Ma3, Quanquan Gu1, Jing Huang2 1Department of Computer Science, University of California, Los Angeles 2JD AI Research, Mountain View, CA 3Department of Computer Science, Stanford University {lingxw,qgu}@cs.ucla.edu {jing.huang,kevin.huang3}@jd.com tengyuma@stanford.edu Abstract First-order meta-learning algorithms have been widely used in practice to learn initial model parameters that can be quickly adapted to new tasks due to their efficiency and effec- tiveness. However, existing studies find that meta-learner can overfit to some specific adap- tation when we have heterogeneous tasks, lead- ing to significantly degraded performance. In Natural Language Processing (NLP) applica- tions, datasets are often diverse and each task has its unique characteristics. Therefore, to ad- dress the overfitting issue when applying first- order meta-learning to NLP applications, we propose to reduce the variance of the gradi- ent estimator used in task adaptation. To this end, we develop a variance-reduced first-order meta-learning algorithm. The core of our al- gorithm is to introduce a novel variance reduc- tion term to the gradient estimation when per- forming the task adaptation. Experiments on two NLP applications: few-shot text classifi- cation and multi-domain dialog state tracking demonstrate the superior performance of our proposed method. 1 Introduction Meta-learning has recently emerged as a promis- ing approach in solving many natural language processing tasks, such as few-shot text classifica- tion (Obamuyide and Vlachos, 2019; Bao et al., 2019), low resource language understanding (Gu et al., 2018; Dou et al., 2019; Yu et al., 2020a), and multi-domain dialogue systems (Qian and Yu, 2019; Huang et al., 2020). In particular, model- agnostic meta-learning (MAML) (Finn et al., 2017), a widely-used meta-learning approach, trains an initial model that can be adapted to a new task with a small number of optimization steps and training data. However, MAML requires the com- putation of second-order derivatives, which can be costly for reinforcement learning and NLP ap- plications. Therefore, numerous computationally- efficient MAML variants (Finn et al., 2017; Li et al., 2017; Nichol et al., 2018; Antoniou et al., 2018; Zintgraf et al., 2019; Song et al., 2020) have been proposed in recent years. First-order meta- learning (Finn et al., 2017; Nichol et al., 2018) is a widely-used method in practice because it is easy to implement, eliminates computationally-intensive second-order derivatives in MAML, and achieves state-of-the-art performance. Although meta-learning including first-order meta-learning has shown promising performances in many applications (Triantafillou et al., 2019), it still somewhat struggles to learn on diverse task dis- tributions (Triantafillou et al., 2020; Rebuffi et al., 2017; Yu et al., 2020c). For first-order meta- learning, it consists of task adaptation and meta updates. Task adaptation aims to obtain a task- specific model for each task by performing several optimization steps based on the current meta model. Then, the meta update aggregates the gradient in- formation of …

Content truncated to 500 words