Constrained Labeled Data Generation for Low-Resource Named Entity Recognition

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4519–4533 August 1–6, 2021. ©2021 Association for Computational Linguistics 4519 Constrained Labeled Data Generation for Low-Resource Named Entity Recognition Ruohao Guo University of Illinois Urbana-Champaign ruohaog2@illinois.edu Dan Roth University of Pennsylvania danroth@seas.upenn.edu Abstract Named Entity Recognition (NER) in low- resource languages has been a long-standing challenge in NLP. Recent work has shown great progress in two directions: develop- ing cross-lingual features/models to transfer knowledge to low-resource languages, and translating source-language training data into low-resource target-language training data by projecting annotations with cheap resources. We focus on the second direction in this study. Existing methods suffer from the low quality of the resulting annotated data in the target lan- guage; for example, they cannot handle word order and lexical ambiguity well. To handle these limitations we propose a novel approach that uses the projected annotation to generate pseudo supervised data with a transformer lan- guage model and a constrained beam search. This allows us to generate more diverse, higher quality, as well as higher quantities of an- notated data in the target language. Experi- ments demonstrate that, when combining our method with available cross-lingual features, it achieves state-of-the-art or competitive perfor- mance on NER in a low-resource setting, espe- cially for languages that are distant from our source language, English. 1 1 Introduction Named entity recognition (NER), the task of find- ing and classifying named entities in text, has been a mature topic in natural language processing (NLP). However, its success is highly dependent on the amount and quality of annotated data. For most of the world’s languages, the amount of supervised resource is limited. How to develop a good NER system with little to no annotated data has become a challenging problem. 1The code and data of the paper are available at: http: //cogcomp.org/page/publication_view/945 (a) (b) Figure 1: (a) A pipeline of our data generation system; (b) An English-to-German example. An NER anno- tated English sentence at the top as the input produces (multiple) NER annotated German sentence(s) at the bottom. Red words are labeled named entities. Our generation method is denoted by CLDG (see §3.3 for details). To address this challenge in low-resource NER, recent works study the benefits of weakly- or partially-annotated data (Dehghani et al., 2018; Mayhew et al., 2019), and that of transferring knowledge from the high-resource languages to the low-resource languages. Common corpora for developing cross-linguality include parallel text (Wang and Manning, 2014; Ni and Florian, 2016), Wikipedia (Nothman et al., 2013; Pan et al., 2017), and multilingual dictionaries or gazetteers (Tsai et al., 2016). However, the effectiveness of these approaches depends on the quality and quantity of data. For example, parallel text in some low- resource languages is unavailable and the dictio- nary size is usually smaller; there are 295 languages in Wikipedia, but most of them are too sparse to be useful. Mayhew et al. (2017) and Xie et al. (2018) employed phrase-level and word-level translation respectively to produce target-language training 4520 data …

Content truncated to 500 words