LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting

Proceedings of the 29th International Conference on Computational Linguistics, pages 2374–2387 October 12–17, 2022. 2374 LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting Xiang Chen1,2∗, Lei Li1,2∗, Shumin Deng1,2, Chuanqi Tan3, Changliang Xu4 Fei Huang3, Luo Si3, Huajun Chen1,2, Ningyu Zhang1,2,† 1Zhejiang University & AZFT Joint Lab for Knowledge Engine, China 2Hangzhou Innovation Center, Zhejiang University, 3Alibaba Group 4State Key Laboratory of Media Convergence Production Technology and Systems {xiang_chen, leili21, 231sm, huajunsir, zhangningyu}@zju.edu.cn, {chuanqi.tcq, f.huang, luo.si}@alibaba-inc.com, xu@shuwen.com Abstract Most NER methods rely on extensive labeled data for model training, which struggles in the low-resource scenarios with limited training data. Existing dominant approaches usually suffer from the challenge that the target do- main has different label sets compared with a resource-rich source domain, which can be concluded as class transfer and domain transfer. In this paper, we propose a lightweight tuning paradigm for low-resource NER via pluggable prompting (LightNER). Specifically, we con- struct the unified learnable verbalizer of entity categories to generate the entity span sequence and entity categories without any label-specific classifiers, thus addressing the class transfer issue. We further propose a pluggable guid- ance module by incorporating learnable param- eters into the self-attention layer as guidance, which can re-modulate the attention and adapt pre-trained weights. Note that we only tune those inserted module with the whole param- eter of the pre-trained language model fixed, thus, making our approach lightweight and flex- ible for low-resource scenarios and can better transfer knowledge across domains. Experi- mental results show that LightNER can obtain comparable performance in the standard super- vised setting and outperform strong baselines in low-resource settings1. 1 Introduction Named Entity Recognition (NER) has been a fun- damental task of research within the Natural Lan- guage Processing (NLP) community. Mostly, the NER task is formulated as a sequence classifica- tion task, aiming to assign the labels to each en- tity in the input sequence. And those entity la- bels are all based on pre-defined categories, such as location, organization, person. The current ∗ Equal Contribution. † Corresponding Author. 1Code is in https://github.com/zjunlp/ DeepKE/tree/main/example/ner/few-shot. News others location rating_average flight_mod Olympic champion Agassi meets Karim Alami of Morocco in the first round. person person organization person location Dole was flanked by several California Republican politicians including Gov. Movie Atis toloc.city_name fromloc.city_name actor Is there a well rated western movie starring Jack Lemmon? What is the shortest flight from Milwaukee to San Diego ? Rich-Resoure Low-Resoure Low-Resoure Domain Transfer person location organi- zation ... rating_ average actor ... flight_ mod ... Class Transfer Figure 1: Examples of NER involved in Class Transfer and Domain Transfer in low-resource setting. mature methodologies for handling NER is us- ing Pre-trained language models (PLMs) (Devlin et al., 2019) equipped with several NER paradigms to perform extensive training process on large corpus, such as label-specific classifier paradigm (LC) (Strubell et al., 2017; Cui and Zhang, 2019), machine reading comprehension paradigm (MRC) (Yu et al., 2020) and unified generative paradigm (BartNER (Yan et al., 2021)). Unfortunately the …

Content truncated to 500 words