MANNER: A Variational Memory-Augmented Model for Cross Domain Few-Shot Named Entity Recognition

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4261–4276 July 9-14, 2023 ©2023 Association for Computational Linguistics MANNER: A Variational Memory-Augmented Model for Cross Domain Few-Shot Named Entity Recognition Jinyuan Fang1∗, Xiaobin Wang2, Zaiqiao Meng1, Pengjun Xie2, Fei Huang2, Yong Jiang2† 1 School of Computing Science, University of Glasgow 2 DAMO Academy, Alibaba Group fangjy6@gmail.com, xuanjie.wxb@alibaba-inc.com, zaiqiao.meng@glasgow.ac.uk, yongjiang.jy@alibaba-inc.com Abstract This paper focuses on the task of cross do- main few-shot named entity recognition (NER), which aims to adapt the knowledge learned from source domain to recognize named en- tities in target domain with only a few labeled examples. To address this challenging task, we propose MANNER, a variational memory- augmented few-shot NER model. Specifically, MANNER uses a memory module to store in- formation from the source domain and then re- trieve relevant information from the memory to augment few-shot tasks in the target domain. In order to effectively utilize the information from memory, MANNER uses optimal transport to retrieve and process information from memory, which can explicitly adapt the retrieved infor- mation from source domain to target domain and improve the performance in the cross do- main few-shot setting. We conduct experiments on both English and Chinese cross domain few- shot NER datasets, and the experimental results demonstrate that MANNER can achieve supe- rior performance 1. 1 Introduction Named Entity Recognition (NER) is a fundamental NLP task that aims at classifying mention spans into entity types. Previous works mainly study the NER task in a supervised setting (Chiu and Nichols, 2016; Devlin et al., 2019; Yamada et al., 2020). However, supervised learning requires large- scale annotated datasets, which can be difficult to obtain in some scenarios (e.g., annotating biomedi- cal named entities always requires domain exper- tise (Ogren et al., 2008)). In this paper, we focus on a more practical and challenging setting in real- world applications, namely cross domain few-shot NER — given a source domain with sufficient la- beled data and a target domain with a few labeled ∗Work done when interning at Alibaba DAMO Academy. †Yong Jiang is the corresponding author. 1 Our code is publicly available at: https://github. com/Alibaba-NLP/MANNER Source Domain NER Model !! %: Jimmy won a medal &: [B-Person] [O] [O] [O] … '$ %: it will snow tomorrow &: [O] [O] [O] [B-Time] … ($ A task in source domain Target Domain %: Einstein was born in German. &: [B-Scientist] [O] [O] [O] [O] '% … finetuning pretraining predict B-Scientist Inputs: Leibniz was a famous mathematician. Figure 1: Illustration of the cross domain few-shot NER task, where the NER model is first pretrained on a set of tasks (each task has a support set, e.g., Ss and a query set, e.g., Qs) in source domain and then adapted to a few-shot task in target domain with a few labeled data. data, the goal is to correctly recognize named enti- ties in the target domain (Hou et al., 2020). This is achieved by adapting the …

Content truncated to 500 words