What Matters for Neural Cross-Lingual Named Entity Recognition: An Empirical Analysis

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 6395–6401, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 6395 What Matters for Neural Cross-Lingual Named Entity Recognition: An Empirical Analysis Xiaolei Huang1∗, Jonathan May2, Nanyun Peng2 1Dept. of Information Science, University of Colorado Boulder 2University of Southern California Information Sciences Institute xiaolei.huang@colorado.edu, {jonmay, npeng}@isi.edu ∗ Abstract Building named entity recognition (NER) models for languages that do not have much training data is a challenging task. While recent work has shown promising results on cross-lingual transfer from high-resource lan- guages to low-resource languages, it is unclear what knowledge is transferred. In this paper, we first propose a simple and efficient neu- ral architecture for cross-lingual NER. Exper- iments show that our model achieves competi- tive performance with the state-of-the-art. We further analyze how transfer learning works for cross-lingual NER on two transferable fac- tors: sequential order and multilingual em- beddings, and investigate how model perfor- mance varies across entity lengths. Finally, we conduct a case-study on a non-Latin lan- guage, Bengali, which suggests that leveraging knowledge from Wikipedia will be a promis- ing direction to further improve the model per- formances. Our results can shed light on future research for improving cross-lingual NER. 1 Introduction Named Entity Recognition (NER) is an important NLP task that identifies the boundary and type of named entities (e.g., person, organization, loca- tion) in texts. However, for some languages, it is hard to obtain enough labeled data to build a fully supervised learning model. Cross-lingual transfer models, which train on high-resource languages and transfer to target languages are a promis- ing direction for languages with little annotated data (Bharadwaj et al., 2016; Tsai et al., 2016; Pan et al., 2017; Yang et al., 2017; Mayhew et al., 2017; Wang et al., 2017; Cotterell and Duh, 2017; Feng et al., 2018; Xie et al., 2018; Zhou et al., 2019). ∗The work was done when the first author worked as an intern at USC ISI. Cross-lingual NER models learn transferable knowledge mainly from four sources: transla- tion (Mayhew et al., 2017; Feng et al., 2018), bilin- gual embeddings (Ni et al., 2017; Xie et al., 2018), phonetic alphabets (Bharadwaj et al., 2016) and multi-source learning (Mayhew et al., 2017; Lin et al., 2018; Zhou et al., 2019; Chen et al., 2019; Rahimi et al., 2019). Translation can be applied at either the sentence level via machine transla- tion or at the word and phrase level via applica- tion of bilingual dictionaries. Bilingual embed- dings are a form of bilingual dictionaries; they constitute a projection from one pre-trained lan- guage representation into the same vector space as the other such that words with the same mean- ing have similar representations (Conneau et al., 2018). Phonetic alphabets enable different lan- guages to share the same pronunciation characters so that the character-level knowledge is transfer- able between languages that otherwise have dif- ferent character …

Content truncated to 500 words