Few-Shot Named Entity Recognition: An Empirical Baseline Study

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10408–10423 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 10408 Few-Shot Named Entity Recognition: An Empirical Baseline Study Jiaxin Huang1 ∗, Chunyuan Li2 †, Krishan Subudhi2, Damien Jose2, Shobana Balakrishnan2, Weizhu Chen2, Baolin Peng2, Jianfeng Gao2, Jiawei Han1 1University of Illinois Urbana-Champaign 2Microsoft {jiaxinh3, hanj}@illinois.edu {chunyl,krkusuk,dajose,shobanab,wzchen,bapeng,jfgao}@microsoft.com Website: https://few-shot-ner-benchmark.github.io/index.html Abstract This paper presents an empirical study to effi- ciently build named entity recognition (NER) systems when a small amount of in-domain labeled data is available. Based upon recent Transformer-based self-supervised pre-trained language models (PLMs), we investigate three orthogonal schemes to improve model gener- alization ability in few-shot settings: (1) meta- learning to construct prototypes for different entity types, (2) task-specific supervised pre- training on noisy web data to extract entity- related representations and (3) self-training to leverage unlabeled in-domain data. On 10 pub- lic NER datasets, we perform extensive empir- ical comparisons over the proposed schemes and their combinations with various propor- tions of labeled data, our experiments show that (i) in the few-shot learning setting, the proposed NER schemes significantly improve or outperform the commonly used baseline, a PLM-based linear classifier fine-tuned using domain labels. (ii) We create new state-of-the- art results on both few-shot and training-free settings compared with existing methods. 1 Introduction Named Entity Recognition (NER) involves pro- cessing unstructured text, locating and classifying named entities (certain occurrences of words or ex- pressions) into particular categories of pre-defined entity types, such as persons, organizations, loca- tions, medical codes, dates and quantities. NER serves as an important first component for tasks such as information extraction (Ritter et al., 2012), information retrieval (Guo et al., 2009), question answering (Moll´a et al., 2006), task-oriented dia- logues (Peng et al., 2020a; Gao et al., 2019) and other language understanding applications (Nadeau and Sekine, 2007; Shaalan, 2014). Deep learning ∗Work performed during an internship at Microsoft † Corresponding author Linear Classifier Fine-tuning Prototype Methods Noisy Supervised Pretraining Self-Training 1 2 3 Figure 1: An overview of methods studied in our paper. Linear classifier fine-tuning is a default baseline that up- dates an NER model from pre-trained Roberta/BERT. We study three orthogonal strategies to improve NER models in the limited labeled data settings. has shown remarkable success in NER in recent years, especially with self-supervised pre-trained language models (PLMs) such as BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019c). State-of-the-art (SoTA) NER models are often ini- tialized with PLM weights, fine-tuned with stan- dard supervised learning. One classic approach is to add a linear classifier on top of the representa- tions provided by PLMs, and fine-tune the entire model using a cross-entropy objective on domain- specific labels (Devlin et al., 2019). Despite its simplicity, the approach generally results in good performance on benchmarks and serves as a strong baseline in this study. Unfortunately, even with these PLMs, building NER systems still remains a labor-intensive, time- consuming task. It requires rich domain knowledge and expert experience to annotate a …

Content truncated to 500 words