COPNER: Contrastive Learning with Prompt Guiding for Few-shot Named Entity Recognition

Proceedings of the 29th International Conference on Computational Linguistics, pages 2515–2527 October 12–17, 2022. 2515 COPNER: Contrastive Learning with Prompt Guiding for Few-shot Named Entity Recognition Yucheng Huang1 Kai He1 Yige Wang1 Xianli Zhang1 Tieliang Gong1 Rui Mao2 Chen Li1∗ 1School of Computer Science and Technology, Xi’an Jiaotong University 2School of Computer Science and Engineering, Nanyang Technological University {huangyucheng, hk52025804, jihejue039, xlbryant}@stu.xjtu.edu.cn {gongtl, cli}@xjtu.edu.cn, rui.mao@ntu.edu.sg Abstract Distance metric learning has become a popular solution for few-shot Named Entity Recogni- tion (NER). The typical setup aims to learn a similarity metric for measuring the seman- tic similarity between test samples and refer- ents, where each referent represents an entity class. The effect of this setup may, however, be compromised for two reasons. First, there is typically a limited optimization exerted on the representations of entity tokens after initing by pre-trained language models. Second, the referents may be far from representing corre- sponding entity classes due to the label scarcity in the few-shot setting. To address these chal- lenges, we propose a novel approach named COntrastive learning with Prompt guiding for few-shot NER (COPNER). We introduce a novel prompt composed of class-specific words to COPNER to serve as 1) supervision signals for conducting contrastive learning to optimize token representations; 2) metric referents for distance-metric inference on test samples. Ex- perimental results demonstrate that COPNER outperforms state-of-the-art models with a sig- nificant margin in most cases. Moreover, COP- NER shows great potential in the zero-shot set- ting. The source code is available at: https: //github.com/AndrewHYC/COPNER. 1 Introduction As a fundamental task in Nature Language Process (NLP), Named Entity Recognition (NER) aims to identify the spans of text according to a pre-defined set of entity classes, such as person, organization and location (Sang and De Meulder, 2003). Many down-stream tasks heavily rely on these extracted entities, such as aspect-level sentiment (Mao and Li, 2021), intention recognition (Vedula et al., 2020), and knowledge graph construction (He et al., 2021). An enormous number of neural methods have shown promising ability on NER tasks (Chiu ∗Corresponding author and Nichols, 2016; Yadav and Bethard, 2018; Li et al., 2020), whereas insufficient labeled data in different domains are still a significant challenging for the community. Considering obtaining full an- notated data is labor-intensive and time-consuming, few-shot NER studies (Fritzler et al., 2019; Yang and Katiyar, 2020; Das et al., 2021; Cui et al., 2021; Ma et al., 2021) are raising more attention, which can alleviate annotation dependence and help neu- ral methods transfer to other tasks easier. Recently, Prompt Learning (PL) becomes a pop- ular technology in NLP and shows great potential for dealing with few-shot issues (Liu et al., 2021b; Ding et al., 2021b; Chen et al., 2021; Mao et al., 2022). Typical prompt learning is designed for understanding sentence-level tasks by decoding a special marker of an input. However, it is challeng- ing to adapt PL to token-level tasks, which needs to identify the class of each token. Inspired by PL, TemplateNER (Cui et al., 2021) applies …

Content truncated to 500 words