Meta-Information Guided Meta-Learning for Few-Shot Relation Classification

Proceedings of the 28th International Conference on Computational Linguistics, pages 1594–1605 Barcelona, Spain (Online), December 8-13, 2020 1594 Meta-Information Guided Meta-Learning for Few-Shot Relation Classification Bowen Dong1∗, Yuan Yao1∗, Ruobing Xie2, Tianyu Gao1, Xu Han1, Zhiyuan Liu1† , Fen Lin2, Leyu Lin2, Maosong Sun1 1Department of Computer Science and Technology Institute for Artificial Intelligence, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology, China 2WeChat Search Application Department, Tencent, China {dongbw18,yuan-yao18}@mails.tsinghua.edu.cn Abstract Few-shot classification requires classifiers to adapt to new classes with only a few training in- stances. State-of-the-art meta-learning approaches such as MAML learn how to initialize and fast adapt parameters from limited instances, which have shown promising results in few-shot classification. However, existing meta-learning models solely rely on implicit instance-based statistics, and thus suffer from instance unreliability and weak interpretability. To solve this problem, we propose a novel meta-information guided meta-learning (MIML) framework, where semantic concepts of classes provide strong guidance for meta-learning in both initialization and adaptation. In effect, our model can establish connections between instance-based information and semantic-based information, which enables more effective initialization and faster adapta- tion. Comprehensive experimental results on few-shot relation classification demonstrate the effectiveness of the proposed framework. Notably, MIML achieves comparable or superior per- formance to humans with only one shot on FewRel evaluation. The source code and experiment details of this paper can be obtained from https://github.com/thunlp/MIML. 1 Introduction Conventional machine learning algorithms, especially neural methods, require an adequate amount of data to learn model parameters. To alleviate the heavy reliance on annotated data, few-shot learning, which aims at adapting to new tasks with only a few training examples, has drawn more and more attention. Few-shot classification is a typical few-shot learning task, which samples several new classes with a handful of training examples (i.e., support instances) and query instances, and requires models to classify these queries into given classes (Lake et al., 2011; Vinyals et al., 2016). To grasp the patterns of new classes with limited examples, meta-learning was proposed. Inspired by human behaviors, meta-learning models focus on learning to learn: they learn how to better initialize parameters and fast adapt classification models from given instances. For example, MAML (Finn et al., 2017) finds the best initialization point of parameters, where it can take minimal efforts to reach the optimal points for each class. To this end, MAML adapts towards each class by gradient steps using support instances, and uses the loss of the adapted model on the query instances to optimize the initialization parameters. However, meta-learning still has three challenges: (1) Most meta-learning methods learn how to learn (i.e., how to initialize and adapt) solely relying on instance statistics, which inevitably suffer from data sparsity and noise in low-resource scenarios, especially in text domain. (2) The approach of learning to learn, like the learning process itself, is a black-box and thus lacks interpretability. (3) Most conventional meta-learning methods are designed for few-shot classification, and cannot well handle zero-shot sce- narios, where no support instances …

Content truncated to 500 words