Better Modeling of Incomplete Annotations for Named Entity Recognition

Proceedings of NAACL-HLT 2019, pages 729–734 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 729 Better Modeling of Incomplete Annotations for Named Entity Recognition Zhanming Jie1, Pengjun Xie2, Wei Lu1, Ruixue Ding2 and Linlin Li2 1StatNLP Research Group, Singapore University of Technology and Design 2DAMO Academy, Alibaba Group zhanming jie@mymail.sutd.edu.sg, luwei@sutd.edu.sg {chengchen.xpj, ada.drx, linyan.lll}@alibaba-inc.com Abstract Supervised approaches to named entity recog- nition (NER) are largely developed based on the assumption that the training data is fully annotated with named entity information. However, in practice, annotated data can often be imperfect with one typical issue being the training data may contain incomplete annota- tions. We highlight several pitfalls associated with learning under such a setup in the con- text of NER and identify limitations associated with existing approaches, proposing a novel yet easy-to-implement approach for recogniz- ing named entities with incomplete data anno- tations. We demonstrate the effectiveness of our approach through extensive experiments.1 1 Introduction Named entity recognition (NER) (Tjong Kim Sang, 2002; Tjong Kim Sang and De Meul- der, 2003) as one of the most fundamental tasks within natural language processing (NLP) has received significant attention. Most existing approaches to NER focused on a supervised setup, where fully annotated named entity information is assumed to be available during the training phase. However, in practice, obtaining high-quality annotations can be a very laborious and expensive process (Snow et al., 2008). One of the common issues with data annotations is there may be incomplete annotations. Figure 1 shows an example sentence with two named entities “John Lloyd Jones” and “BBC ra- dio” of type PER (person) and ORG (organization), respectively. Following the standard BIOES tag- ging scheme (Ramshaw and Marcus, 1999; Rati- nov and Roth, 2009), the corresponding gold la- bel sequence is shown below the sentence. When the data annotations are incomplete, certain labels 1Our code and data are available at http://statnlp. org/research/ie. Sentence: Chairman John Lloyd Jones said on BBC radio Gold: O BPER IPER EPER O O BORG EORG A.1: O BPER - - O - - EORG (Fernandes and Brefeld, 2011) A.2: O BPER IPER EPER O O - - (Carlson et al., 2009) A.3: - BPER IPER EPER - - - - Our assumption Figure 1: An example sentence with gold named entity annotations and different assumptions (i.e., A.1 to A.3) on available labels. “-” represents a missing label. may be missing from the label sequence. Prop- erly defining the task is important, and we argue there are two possible potential pitfalls associated with modeling incomplete annotations, especially for the NER task. Several previous approaches assume the incom- plete annotations can be obtained by simply re- moving either word-level labels (Fernandes and Brefeld, 2011) or span-level labels (Carlson et al., 2009). As shown in Figure 1, under both assump- tions (i.e., A.1 and A.2), there will be words an- notated with O labels. The former approach may even lead to sub-entity level annotations (e.g., “ra- dio” is annotated as part of …

Content truncated to 500 words