PCBERT: Parent and Child BERT for Chinese Few-shot NER

Proceedings of the 29th International Conference on Computational Linguistics, pages 2199–2209 October 12–17, 2022. 2199 PCBERT: Parent and Child BERT for Chinese Few-shot NER Peichao Lai Feiyang Ye Lin Zhang Zhiwei Chen Yanggeng Fu Yingjie Wu Yilei Wang College of Computer and Data Science, Fuzhou University, China yilei@fzu.edu.cn Abstract Achieving good performance on few-shot or zero-shot datasets has been a long-term chal- lenge for NER. The conventional semantic transfer approaches on NER will decrease model performance when the semantic distri- bution is quite different, especially in Chinese few-shot NER. Recently, prompt-tuning has been thoroughly considered for low-resource tasks. But there is no effective prompt-tuning approach for Chinese few-shot NER. In this work, we propose a prompt-based Parent and Child BERT (PCBERT) for Chinese few-shot NER. To train an annotating model on high- resource datasets and then discover more im- plicit labels on low-resource datasets. We fur- ther design a label extension strategy to achieve label transferring from high-resource datasets. We evaluated our model on Weibo and the other three sampling Chinese NER datasets, and the experimental result demonstrates our approach’s effectiveness in few-shot learning. 1 Introduction NER is a fine-grained sequence labeling task, a slight change in each token will significantly im- pact the model results. A big challenge of NER is to enhance the performance in low-resource sce- narios. There are some prior works (Yang et al., 2017; Lee et al., 2017; Abhishek et al., 2017) that demonstrate that transfer learning can improve the model performance. However, they all rely on simi- lar semantic distribution between source and target datasets, and both datasets should contain rich an- notated data. A significant difficulty of few-shot or zero-shot NER is the lack of annotated labels in practical application. Another challenge of Chi- nese NER is the implicit word boundary, which makes it difficult for the model to distinguish the entity boundary. The lexicon-based approach is a standard solution to solve the above issue. But the performance of traditional lexicon-based models in Chinese few-shot NER is still unsatisfactory. Recently, prompt-tuning (Lester et al., 2021) on the pre-trained language models (PLMs) has been thoroughly considered for low-resource scenarios because the prompt-tuning process is highly con- sistent with the target task. Previous work (Cui et al., 2021; Ma et al., 2022; Chen et al., 2021) has demonstrated that prompt-tuning can more effec- tively enhance the model performance on few-shot NER compared with fine-tuning. However, when the semantic distribution is quite different, using prompt-tuning for semantic transfer learning will decrease model performance, which implies the se- mantic transfer is unsuitable for NER in the above situation. Besides, the implicit boundaries of Chi- nese words make the size of the prompt template uncertain and require a higher ability to judge its boundary. Moreover, using inappropriate prompt construction engineering on Chinese few-shot NER datasets can not improve model performance effec- tively but increases training time. In this work, we introduce an enhanced lexicon feature and a prompt-based label transfer approach to address the above issues. We leverage …

Content truncated to 500 words