Cross-lingual Multi-Level Adversarial Transfer to Enhance Low-Resource Name Tagging

Proceedings of NAACL-HLT 2019, pages 3823–3833 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3823 Cross-lingual Multi-Level Adversarial Transfer to Enhance Low-Resource Name Tagging Lifu Huang1, Heng Ji1, Jonathan May2 1 Computer Science Department, Rensselaer Polytechnic Institute {huangl7,jih}@rpi.edu 2 Information Sciences Institute, University of Southern California jonmay@isi.edu Abstract We focus on improving name tagging for low- resource languages using annotations from re- lated languages. Previous studies either di- rectly project annotations from a source lan- guage to a target language using cross-lingual representations or use a shared encoder in a multitask network to transfer knowledge. These approaches inevitably introduce noise to the target language annotation due to mis- matched source-target sentence structures. To effectively transfer the resources, we develop a new neural architecture that leverages multi- level adversarial transfer: (1) word-level ad- versarial training, which projects source lan- guage words into the same semantic space as those of the target language without using any parallel corpora or bilingual gazetteers, and (2) sentence-level adversarial training, which yields language-agnostic sequential features. Our neural architecture outperforms previous approaches on CoNLL data sets. Moreover, on 10 low-resource languages, our approach achieves up to 16% absolute F-score gain over all high-performing baselines on cross-lingual transfer without using any target-language re- sources.1 1 Introduction Low-resource language name tagging is an impor- tant but challenging task. An effective solution is to perform cross-lingual transfer, by leveraging the annotations from high-resource languages. Most of these efforts achieve cross-lingual annotation projection based on bilingual parallel corpora com- bining with automatic word alignment (Yarowsky et al., 2001; Wang et al., 2013; Fang and Cohn, 2016; Ehrmann et al., 2011; Ni et al., 2017), bilingual gazetteers (Feng et al., 2017; Zirikly 1Our programs will be released at https://github. com/wilburOne/AdversarialNameTagger and Hagiwara, 2015), cross-lingual word embed- ding (Fang and Cohn, 2017; Wang et al., 2017; Huang et al., 2018), or cross-lingual Wikifica- tion (Kim et al., 2012; Nothman et al., 2013; Tsai et al., 2016; Pan et al., 2017), but these resources are still only available for dozens of languages. Recent efforts on multi-task learning model each language as one single task while all the tasks share the same encoding layer (Yang et al., 2016, 2017; Lin et al., 2018). These methods can transfer knowledge via the shared encoder without using bilingual resources. However, different languages usually have different underlying sequence struc- tures, as shown in Figure 1. Without an explicit constraint, the encoder is not guaranteed to extract language-independent sequential features. More- over, when the size of annotated resources is not balanced, the encoder is likely to be biased toward the resource-dominant language. NED: ENG: ESP: The European Union5' s competition policy3 has been of central importance4 since European integration2 began1. La política de competencia3 de la Unión Europea5 ha sido de central importancia4 desde que se inició1 la integración europea2. Sedert het begin1 van de Europese integratie2 is het mededingingsbeleid3 van groot belang4 voor de Europese Unie5. Figure 1: Example of parallel sentences between En- …

Content truncated to 500 words