CONTaiNER: Few-Shot Named Entity Recognition via Contrastive Learning

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6338 - 6353 May 22-27, 2022 c⃝2022 Association for Computational Linguistics CONTAINER: Few-Shot Named Entity Recognition via Contrastive Learning Sarkar Snigdha Sarathi Das, Arzoo Katiyar, Rebecca J. Passonneau, Rui Zhang Pennsylvania State University {snigdha, arzoo, rjp49, rmz5227}@psu.edu Abstract Named Entity Recognition (NER) in Few-Shot setting is imperative for entity tagging in low resource domains. Existing approaches only learn class-specific semantic features and inter- mediate representations from source domains. This affects generalizability to unseen target domains, resulting in suboptimal performances. To this end, we present CONTAINER, a novel contrastive learning technique that optimizes the inter-token distribution distance for Few- Shot NER. Instead of optimizing class-specific attributes, CONTAINER optimizes a general- ized objective of differentiating between token categories based on their Gaussian-distributed embeddings. This effectively alleviates overfit- ting issues originating from training domains. Our experiments in several traditional test do- mains (OntoNotes, CoNLL’03, WNUT ’17, GUM) and a new large scale Few-Shot NER dataset (Few-NERD) demonstrate that, on aver- age, CONTAINER outperforms previous meth- ods by 3%-13% absolute F1 points while showing consistent performance trends, even in challenging scenarios where previous ap- proaches could not achieve appreciable perfor- mance. The source code of CONTAINER will be available at: https://github.com/ psunlpgroup/CONTaiNER. 1 Introduction Named Entity Recognition (NER) is a fundamental NLU task that recognizes mention spans in un- structured text and categorizes them into a pre- defined set of entity classes. In spite of its chal- lenging nature, recent deep-learning based ap- proaches (Huang et al., 2015; Ma and Hovy, 2016; Lample et al., 2016; Peters et al., 2018; Devlin et al., 2018) have achieved impressive performance. As these supervised NER models require large-scale human-annotated datasets, few-shot techniques that can effectively perform NER in resource constraint settings have recently garnered a lot of attention. The Islands lie 860 km east of Christchurch O PLACE O QTY. QTY. O O PLACE Attraction Repulsion Figure 1: Contrastive learning dynamics of a token (Is- lands) with all other tokens in an example sentence from GUM (Zeldes, 2017). CONTAINER decreases the em- bedding distance between tokens of the same category (PLACE) while increasing the distance between differ- ent categories ( QTY. and O). Few-shot learning involves learning unseen classes from very few labeled examples (Fei-Fei et al., 2006; Lake et al., 2011; Bao et al., 2020). To avoid overfitting with the limited available data, meta-learning has been introduced to focus on how to learn (Vinyals et al., 2016; Bao et al., 2020). Snell et al. (2017) proposed Prototypical Networks to learn a metric space where the examples of a specific unknown class cluster around a single pro- totype. Although it was primarily deployed in com- puter vision, Fritzler et al. (2019) and Hou et al. (2020) also used Prototypical Networks for few- shot NER. Yang and Katiyar (2020), on the other hand, proposed a supervised NER model that learns class-specific features and extends the intermediate representations to unseen domains. Additionally, …

Content truncated to 500 words