Learning In-context Learning for Named Entity Recognition

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 13661–13675 July 9-14, 2023 ©2023 Association for Computational Linguistics Learning In-context Learning for Named Entity Recognition Jiawei Chen1,4,∗, Yaojie Lu1,†, Hongyu Lin1, Jie Lou3, Wei Jia3, Dai Dai3, Hua Wu3, Boxi Cao1,4, Xianpei Han1,2,†, Le Sun1,2 1Chinese Information Processing Laboratory 2State Key Laboratory of Computer Science Institute of Software, Chinese Academy of Sciences, Beijing, China 3Baidu Inc., Beijing, China 4University of Chinese Academy of Sciences, Beijing, China {jiawei2020,yaojie,hongyu,boxi2020,xianpei,sunle}@iscas.ac.cn {loujie,jiawei07,daidai,wu_hua}@baidu.com Abstract Named entity recognition in real-world ap- plications suffers from the diversity of en- tity types, the emergence of new entity types, and the lack of high-quality annota- tions. To address the above problems, this paper proposes an in-context learning-based NER approach, which can effectively inject in-context NER ability into PLMs and recog- nize entities of novel types on-the-fly using only a few demonstrative instances. Specif- ically, we model PLMs as a meta-function λinstruction, demonstrations, text.M1, and a new en- tity extractor can be implicitly constructed by applying new instruction and demonstrations to PLMs, i.e., (λ.M)(instruction, demonstra- tions) → F where F will be a new entity ex- tractor, i.e., F: text → entities. To inject the above in-context NER ability into PLMs, we propose a meta-function pre-training algorithm, which pre-trains PLMs by comparing the (in- struction, demonstration)-initialized extractor with a surrogate golden extractor. Experimental results on 4 few-shot NER datasets show that our method can effectively inject in-context NER ability into PLMs and significantly out- performs the PLMs+fine-tuning counterparts. 1 Introduction Named entity recognition (NER) aims to detect and classify named entities in text, such as Peo- ple, Disease, and Movie. Traditional NER meth- ods (Lample et al., 2016; Li et al., 2020c; Yan et al., 2021) have achieved remarkable success ∗This work was partially done when Jiawei Chen interned at Baidu. †Corresponding authors. 1This paper represents functions using lambda- calculus (Barendregt, 1992), and each function is represented as λx,y,z.M, where x, y, z are variables and M is function definition/abstraction. The function can apply to arguments such as (λx,y,z.M)(x = A, y = B, z = C) (fully applied) or (λx,y,z.M)(x = A, y = B) (partially applied). Entities: SARS-CoV-2 is virus. COVID-19 is disease. PLMs as meta- function 𝓕: text → [disease, virus] 𝓕: text → [movie] Target types: disease, virus Text: Cancer is a leading … Entities: Cancer is disease. Text: Rabies virus is estimated … Entities: Rabies virus is virus. Text: SARS-CoV-2 is a strain of coronavirus that causes COVID-19. Target types: Disease, Virus Cancer is a leading …-> (Cancer, Disease) Rabies virus is … -> (Rabies virus, Virus) Target types: Movie Avatar is a 2009 epic…-> (Avatar, Movie) Citizen Kane is a … -> (Citizen Kane, Movie) Meta-function In-context NER Extraction Function Extractions Figure 1: Illustration of in-context NER, which uses instruction, demonstrations, and text as input to identify entities. The in-context learning model can be regarded as a meta-function that takes instruction and demonstra- tions as input …

Content truncated to 500 words