Few-NERD: A Few-shot Named Entity Recognition Dataset

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 3198–3213 August 1–6, 2021. ©2021 Association for Computational Linguistics 3198 FEW-NERD: A Few-shot Named Entity Recognition Dataset Ning Ding1,3∗, Guangwei Xu2∗, Yulin Chen3∗, Xiaobin Wang2, Xu Han1, Pengjun Xie2, Hai-Tao Zheng3†, Zhiyuan Liu1† 1Department of Computer Science and Technology, Tsinghua University 2Alibaba Group, 3Shenzhen International Graduate School, Tsinghua University {dingn18, yl-chen17, hanxu17}@mails.tsinghua.edu.cn {kunka.xgw, xuanjie.wxb, chengchen.xpj}@alibaba-inc.com {zheng.haitao}@sz.tsinghua.edu.cn, {liuzy}@tsinghua.edu.cn https://ningding97.github.io/fewnerd/ Abstract Recently, considerable literature has grown up around the theme of few-shot named entity recognition (NER), but little published bench- mark data specifically focused on the practical and challenging task. Current approaches col- lect existing supervised NER datasets and re- organize them into the few-shot setting for em- pirical study. These strategies conventionally aim to recognize coarse-grained entity types with few examples, while in practice, most unseen entity types are fine-grained. In this paper, we present FEW-NERD, a large-scale human-annotated few-shot NER dataset with a hierarchy of 8 coarse-grained and 66 fine- grained entity types. FEW-NERD consists of 188,238 sentences from Wikipedia, 4,601,160 words are included and each is annotated as context or a part of a two-level entity type. To the best of our knowledge, this is the first few-shot NER dataset and the largest human- crafted NER dataset. We construct bench- mark tasks with different emphases to com- prehensively assess the generalization capabil- ity of models. Extensive empirical results and analysis show that FEW-NERD is challeng- ing and the problem requires further research. We make FEW-NERD public at https:// ningding97.github.io/fewnerd/. 1 1 Introduction Named entity recognition (NER), as a fundamental task in information extraction, aims to locate and classify named entities from unstructured natural language. A considerable number of approaches equipped with deep neural networks have shown promising performance (Chiu and Nichols, 2016) on fully supervised NER. Notably, pre-trained lan- guage models (e.g., BERT (Devlin et al., 2019a)) ∗ equal contributions † corresponding authors 1The baselines are available at https://github. com/thunlp/Few-NERD Organization Location Person Art Building Event MISC Product Airport War GPE Bodiesofwater Island Company Education Government Media Other Politicalparty Astronomy Actor Artist/author Athlete Director Other Airplane Broadcast Film Music Disaster Election Mountain Religion ShowORG Sportsleague Award Biology Chemical Currency Disease Car Food Other Painting Hospital Hotel Library Other Other Park Transit Sportsteam EDUdegree God Language Law Livingthing Medical Politician Scholar Soldier Game Other Ship Software Train Figure 1: An overview of FEW-NERD. The inner cir- cle represents the coarse-grained entity types and the outer circle represents the fine-grained entity types, some types are denoted by abbreviations. with an additional classifier achieve significant suc- cess on this task and gradually become the base paradigm. Such studies demonstrate that deep mod- els could yield remarkable results accompanied by a large amount of annotated corpora. With the emerging of knowledge from various domains, named entities, especially ones that need professional knowledge to understand, are diffi- cult to be manually annotated on a large scale. Under this circumstance, studying NER …

Content truncated to 500 words