SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3466 - 3476 December 7-11, 2022 ©2022 Association for Computational Linguistics SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition Jianing Wang1, Chengyu Wang2, Chuanqi Tan2, Minghui Qiu2, Songfang Huang2, Jun Huang2, Ming Gao1,3∗ 1 School of Data Science and Engineering, East China Normal University, Shanghai, China 2 Alibaba Group, Hangzhou, China 3 KLATASDS-MOE, School of Statistics, East China Normal University, Shanghai, China lygwjn@gmail.com, {chengyu.wcy,chuanqi.tcq,minghui.qmh}@alibaba-inc.com, {songfang.hsf,huangjun.hj}@alibaba-inc.com, mgao@dase.ecnu.edu.cn Abstract Few-shot Named Entity Recognition (NER) aims to identify named entities with very lit- tle annotated data. Previous methods solve this problem based on token-wise classification, which ignores the information of entity bound- aries, and inevitably the performance is affected by the massive non-entity tokens. To this end, we propose a seminal span-based prototypical network (SpanProto) that tackles few-shot NER via a two-stage approach, including span ex- traction and mention classification. In the span extraction stage, we transform the sequential tags into a global boundary matrix, enabling the model to focus on the explicit boundary infor- mation. For mention classification, we leverage prototypical learning to capture the semantic representations for each labeled span and make the model better adapt to novel-class entities. To further improve the model performance, we split out the false positives generated by the span extractor but not labeled in the current episode set, and then present a margin-based loss to separate them from each prototype re- gion. Experiments over multiple benchmarks demonstrate that our model outperforms strong baselines by a large margin. 1 1 Introduction Named Entity Recognition (NER) is one of the cru- cial tasks in natural language processing (NLP), which aims at extracting mention spans and clas- sifying them into a set of pre-defined entity type classes. Previous methods (Huang et al., 2015; Santoro et al., 2016; Ma and Hovy, 2016; Lample et al., 2016; Peters et al., 2017) present multiple ∗ Corresponding author. 1All the codes and datasets will be released to the EasyNLP framework (Wang et al., 2022a). URL: https://github. com/alibaba/EasyNLP Support Set Query Example Target Types: PER, LOC Support Examples: Query Example: Outputs: PER: James LOC: Akron, Ohio A record 66.6 million tourists visit- ed [New York City]LOC. [James Francis Cameron]PER was born on August 16, 1954. James played basketball for St. Vincent–St. Mary High School in his hometown of Akron, Ohio. Figure 1: An example of the 2-way 1-shot NER problem. Given a support set with 2 target types where each type is associated with 1 labeled entity, the task is to identify entities in the query example. deep neural architectures and achieve impressive performance. Yet, these conventional approaches heavily depend on the time-consuming and labor- intensive process of data annotation. Thus, an attractive problem of few-shot NER (Ding et al., 2021; Huang et al., 2021; Ma et al., 2022b) has been introduced which involves recognizing novel- class entities based on very few labeled examples (i.e. support examples). An example of the 2-way 1-shot NER problem is shown …

Content truncated to 500 words