Decomposed Meta-Learning for Few-Shot Named Entity Recognition

Findings of the Association for Computational Linguistics: ACL 2022, pages 1584 - 1596 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Decomposed Meta-Learning for Few-Shot Named Entity Recognition Tingting Ma1†∗, Huiqiang Jiang2∗, Qianhui Wu2∗, Tiejun Zhao1, Chin-Yew Lin2 1Harbin Institute of Technology, Harbin, China 2Microsoft Research Asia hittingtingma@gmail.com {hjiang, qianhuiwu, cyl}@microsoft.com, tjzhao@hit.edu.cn Abstract Few-shot named entity recognition (NER) sys- tems aim at recognizing novel-class named en- tities based on only a few labeled examples. In this paper, we present a decomposed meta- learning approach which addresses the problem of few-shot NER by sequentially tackling few- shot span detection and few-shot entity typing using meta-learning. In particular, we take the few-shot span detection as a sequence labeling problem and train the span detector by introduc- ing the model-agnostic meta-learning (MAML) algorithm to find a good model parameter ini- tialization that could fast adapt to new entity classes. For few-shot entity typing, we pro- pose MAML-ProtoNet, i.e., MAML-enhanced prototypical networks to find a good embed- ding space that can better distinguish text span representations from different entity classes. Extensive experiments on various benchmarks show that our approach achieves superior per- formance over prior methods.1 1 Introduction Named entity recognition (NER) aims at locating and classifying text spans into pre-defined entity classes such as locations, organizations, etc. Deep neural architectures have shown great success in fully supervised NER (Lample et al., 2016; Ma and Hovy, 2016; Chiu and Nichols, 2016; Peters et al., 2017) with a fair amount of labeled data available for training. However, in practical applications, NER systems are usually expected to rapidly adapt to some new entity types unseen during training. It is costly while not flexible to collect a number of additional labeled data for these types. As a result, the problem of few-shot NER, which in- volves learning unseen entity types from only a *Equal contributions. †Work during internship at Microsoft Research Asia. 1Our implementation is publicly available at https: //github.com/microsoft/vert-papers/tree/ master/papers/DecomposedMetaNER few labeled examples for each class (also known as support examples), has attracted considerable attention from the research community in recent years. Previous studies on few-shot NER are typically based on token-level metric learning, in which a model compares each query token to the proto- type (Snell et al., 2017) of each entity class or each token of support examples and assign the label ac- cording to their distances (Fritzler et al., 2019; Hou et al., 2020; Yang and Katiyar, 2020). Alterna- tively, some more recent attempts have switched to span-level metric-learning (Yu et al., 2021; Wang et al., 2021a) to bypass the issue of token-wise la- bel dependency while explicitly utilizing phrasal representations. However, these methods based on metric learn- ing might be less effective when encountering large domain gap, since they just directly use the learned metric without any further adaptation to the target domain. In other words, they do not fully explore the information brought by the support examples. There also exist additional limitations in the current methods based on span-level metric learning. …

Content truncated to 500 words