A Little Annotation does a Lot of Good: A Study in Bootstrapping Low-resource Named Entity Recognizers

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5164–5174, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5164 A Little Annotation does a Lot of Good: A Study in Bootstrapping Low-resource Named Entity Recognizers Aditi Chaudhary, Jiateng Xie, Zaid Sheikh, Graham Neubig, Jaime G. Carbonell {aschaudh, jiatengx, zsheikh, gneubig, jgc}@cs.cmu.edu Language Technologies Institute Carnegie Mellon University Abstract Most state-of-the-art models for named en- tity recognition (NER) rely on the availabil- ity of large amounts of labeled data, mak- ing them challenging to extend to new, lower- resourced languages. However, there are now several proposed approaches involving either cross-lingual transfer learning, which learns from other highly resourced languages, or ac- tive learning, which efficiently selects effec- tive training data based on model predictions. This paper poses the question: given this re- cent progress, and limited human annotation, what is the most effective method for effi- ciently creating high-quality entity recognizers in under-resourced languages? Based on ex- tensive experimentation using both simulated and real human annotation, we find a dual- strategy approach best, starting with a cross- lingual transferred model, then performing tar- geted annotation of only uncertain entity spans in the target language, minimizing annotator effort. Results demonstrate that cross-lingual transfer is a powerful tool when very little data can be annotated, but an entity-targeted anno- tation strategy can achieve competitive accu- racy quickly, with just one-tenth of training data. The code is publicly available here.1 1 Introduction Named entity recognition (NER) is the task of de- tecting and classifying named entities in text into a fixed set of pre-defined categories (person, loca- tion, etc.) with several downstream applications including machine reading (Chen et al., 2017), en- tity and event co-reference (Yang and Mitchell, 2016), and text mining (Han and Sun, 2012). Re- cent advances in deep learning have yielded state- of-the-art performance on many sequence label- ing tasks, including NER (Collobert et al., 2011; 1https://github.com/Aditi138/ EntityTargetedActiveLearning Ma and Hovy, 2016; Lample et al., 2016; Pe- ters et al., 2018). However, the performance of these models is highly dependent on the availabil- ity of large amounts of annotated data, and as a result their accuracy is significantly lower on lan- guages that have fewer resources than English. In this work, we ask the question “how can we effi- ciently bootstrap a high-quality named entity rec- ognizer for a low-resource language with only a small amount of human effort?” Specifically, we leverage recent advances in data-efficient learning for low-resource languages, proposing the follow- ing “recipe” for bootstrapping low-resource entity recognizers: First, we use cross-lingual transfer learning (Yarowsky et al., 2001; Ammar et al., 2016), which applies a model trained on another language to low-resource languages, to provide a good preliminary model to start the bootstrap- ping process. Specifically, we use the model of Xie et al. (2018), which reports strong results on a number of language pairs. Next, on top of this transferred model …

Content truncated to 500 words