Template-Based Named Entity Recognition Using BART

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1835–1845 August 1–6, 2021. ©2021 Association for Computational Linguistics 1835 Template-Based Named Entity Recognition Using BART Leyang Cui†3♥, Yu Wu‡, Jian Liu3♥, Sen Yang3♥, Yue Zhang3♥ ∗ †Zhejiang University ‡Microsoft Research Asia 3School of Engineering, Westlake University ♥Institute of Advanced Technology, Westlake Institute of Advanced Study {cuileyang,liujian,yangsen,zhangyue}@westlake.edu.cn Wu.Yu@microsoft.com Abstract There is a recent interest in investigating few- shot NER, where the low-resource target do- main has different label sets compared with a resource-rich source domain. Existing meth- ods use a similarity-based metric. However, they cannot make full use of knowledge trans- fer in NER model parameters. To address the issue, we propose a template-based method for NER, treating NER as a language model ranking problem in a sequence-to-sequence framework, where original sentences and state- ment templates filled by candidate named en- tity span are regarded as the source sequence and the target sequence, respectively. For in- ference, the model is required to classify each candidate span based on the corresponding template scores. Our experiments demonstrate that the proposed method achieves 92.55% F1 score on the CoNLL03 (rich-resource task), and significantly better than fine-tuning BERT 10.88%, 15.34%, and 11.73% F1 score on the MIT Movie, the MIT Restaurant, and the ATIS (low-resource task), respectively. 1 Introduction Named entity recognition (NER) is a fundamental task in natural language processing, which identi- fies mention spans from text inputs according to pre-defined entity categories (Tjong Kim Sang and De Meulder, 2003), such as location, person, or- ganization, etc. The current dominant methods use a sequential neural network such as BiLSTM (Hochreiter and Schmidhuber, 1997) and BERT (Devlin et al., 2019) is used to represent the input text, and softmax (Chiu and Nichols, 2016; Strubell et al., 2017; Cui and Zhang, 2019) or CRF (Lample et al., 2016; Ma and Hovy, 2016; Luo et al., 2020) output layers to assign named entity tags (e.g. or- ganization, person and location) or non-entity tags ∗Corresponding Author Who played jack in Nightmare Before Christmas title character All I want is salmon dish Only France and Britain backed Fischler’s proposal person location location Restaurant News Movie Figure 1: Example of NER on different domains. on each input token. Such a system is illustrated in Figure 2(a). Neural NER models require large labeled train- ing data, which can be available for certain domains such as news, but scarce in most other domains. Ideally, it would be desirable to transfer knowl- edge from the resource-rich news domain so that a model can be used in target domains based on a few labeled instances. In practice, however, a challenge is that entity categories can be different across different domains. As shown in Figure 1, the system is required to identify location and per- son in the news domain, but character and title in the movie domain. Both a softmax layer and CRF layer require a consistent label set between training and testing. As a result, given a new target domain, the output …

Content truncated to 500 words