Label Semantics for Few Shot Named Entity Recognition

Findings of the Association for Computational Linguistics: ACL 2022, pages 1956 - 1971 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Label Semantics for Few Shot Named Entity Recognition Jie Ma1 Miguel Ballesteros1 Srikanth Doss1 Rishita Anubhai1 Sunil Mallya1∗ Yaser Al-Onaizan1∗ Dan Roth1,2 1AWS AI Labs 2Computer and Information Science, University of Pennsylvania {jieman, ballemig, srikad, ranubhai, drot}@amazon.com mallya16@gmail.com, onaizan2000@yahoo.com Abstract We study the problem of few shot learning for named entity recognition. Specifically, we leverage the semantic information in the names of the labels as a way of giving the model addi- tional signal and enriched priors. We propose a neural architecture that consists of two BERT encoders, one to encode the document and its tokens and another one to encode each of the labels in natural language format. Our model learns to match the representations of named entities computed by the first encoder with la- bel representations computed by the second encoder. The label semantics signal is shown to support improved state-of-the-art results in multiple few shot NER benchmarks and on- par performance in standard benchmarks. Our model is especially effective in low resource settings. 1 Introduction Named entity recognition (NER) seeks to locate named entity spans in unstructured text and clas- sify them into pre-defined categories such as PER- SON, LOCATION and ORGANIZATION (Tjong Kim Sang and De Meulder, 2003a). As a funda- mental natural language understanding task, NER often serves as an upstream component for more complex tasks such as question answering (Mollá et al., 2006), relation extraction (Chan and Roth, 2011) and coreference resolution (Clark and Man- ning, 2015). However, building an accurate NER system has traditionally required large amounts of high quality annotated in-domain data (Lison et al., 2020; Chen et al., 2020). This usually involves well defined annotation guidelines and training of annotators, which requires rich domain knowledge and can be prohibitively expensive (Huang et al., 2020). ∗Work done while at AWS AI Labs. Few shot learning (FSL) (Vinyals et al., 2017; Finn et al., 2017; Snell et al., 2017) aims at per- forming a task using only very few annotated ex- amples (i.e. support set). Similarity-based methods, such as prototypical networks, are extensively studied and show great success for FSL (Vinyals et al., 2017; Snell et al., 2017; Yu et al., 2018a; Hou et al., 2020). The core idea is to classify input examples from a new domain based on their similarities with representa- tions of each class in the support set. These meth- ods do not utilize the semantics of label names and usually represent labels by directly averaging the embedding of support set examples, oversimpli- fying the learning of label representations. The main premise of our work is that label names carry meaning that our models can induce from data; the labels are themselves words that appear in text in various contexts and are thus semantically re- lated to other words that appear in text, and this relatedness can be leveraged. For example, the representation of “Lionel Messi” is more …

Content truncated to 500 words