Named Entity Recognition with Partially Annotated Training Data

Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 645–655 Hong Kong, China, November 3-4, 2019. c⃝2019 Association for Computational Linguistics 645 Named Entity Recognition with Partially Annotated Training Data Stephen Mayhew♮, Snigdha Chaturvedi♯, Chen-Tse Tsai♭, Dan Roth♮ ♮University of Pennsylvania, Philadelphia, PA, 19104 ♯University of North Carolina, Chapel Hill, NC, 27599 ♭Bloomberg LP {mayhew, danroth}@seas.upenn.edu, snigdha@cs.unc.edu, ctsai54@bloomberg.net Abstract Supervised machine learning assumes the availability of fully-labeled data, but in many cases, such as low-resource languages, the only data available is partially annotated. We study the problem of Named Entity Recogni- tion (NER) with partially annotated training data in which a fraction of the named enti- ties are labeled, and all other tokens, entities or otherwise, are labeled as non-entity by default. In order to train on this noisy dataset, we need to distinguish between the true and false neg- atives. To this end, we introduce a constraint- driven iterative algorithm that learns to detect false negatives in the noisy set and downweigh them, resulting in a weighted training set. With this set, we train a weighted NER model. We evaluate our algorithm with weighted vari- ants of neural and non-neural NER models on data in 8 languages from several language and script families, showing strong ability to learn from partial data. Finally, to show real-world efficacy, we evaluate on a Bengali NER corpus annotated by non-speakers, outperforming the prior state-of-the-art by over 5 points F1. 1 Introduction Most modern approaches to NLP tasks rely on super- vised learning algorithms to learn and generalize from labeled training data. While this has proven successful in high-resource scenarios, this is not realistic in many cases, such as low-resource languages, as the required amount of training data just doesn’t exist. However, partial annotations are often easy to gather. We study the problem of using partial annotations to train a Named Entity Recognition (NER) system. In this setting, all (or most) identified entities are correct, but not all entities have been identified, and crucially, there are no reliable examples of the negative class. The sentence shown in Figure 1 shows examples of both a gold and a partially annotated sentence. Such partially annotated data is relatively easy to obtain: for Figure 1: This example has three entities: Arsenal, Unai Emery, and Arsene Wenger. In the Partial row, the situation addressed in this paper, only the first and last are tagged, and all other tokens are assumed to be non-entities, making Unai Emery a false negative as compared to Gold. Our model is an iteratively learned binary classifier used to assign weights to each token indicating its chances of being correctly labeled. The Oracle row shows optimal weights. example, a human annotator who does not speak the target language may recognize common entities, but not uncommon ones. With no reliable examples of the negative class, the problem becomes one of estimating which unlabeled instances are true negatives and which are false negatives. To address the above-mentioned challenge, we present Constrained Binary Learning (CBL) …

Content truncated to 500 words