Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Language

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6505–6514 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6505 Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Language Qianhui Wu1, Zijia Lin2, B¨orje F. Karlsson2, Jian-Guang Lou2, and Biqing Huang1 1Beijing National Research Center for Information Science and Technology (BNRist) Department of Automation, Tsinghua University, Beijing 100084, China wuqianhui@tsinghua.org.cn, hbq@tsinghua.edu.cn 2Microsoft Research, Beijing 100080, China {zijlin,borje.karlsson,jlou}@microsoft.com Abstract To better tackle the named entity recognition (NER) problem on languages with little/no la- beled data, cross-lingual NER must effectively leverage knowledge learned from source lan- guages with rich labeled data. Previous works on cross-lingual NER are mostly based on la- bel projection with pairwise texts or direct model transfer. However, such methods ei- ther are not applicable if the labeled data in the source languages is unavailable, or do not leverage information contained in unlabeled data in the target language. In this paper, we propose a teacher-student learning method to address such limitations, where NER models in the source languages are used as teachers to train a student model on unlabeled data in the target language. The proposed method works for both single-source and multi-source cross- lingual NER. For the latter, we further propose a similarity measuring method to better weight the supervision from different teacher models. Extensive experiments for 3 target languages on benchmark datasets well demonstrate that our method outperforms existing state-of-the- art methods for both single-source and multi- source cross-lingual NER. 1 Introduction Named entity recognition (NER) is the task of identifying text spans that belong to pre-defined categories, like locations, person names, etc. It’s a fundamental component in many downstream tasks, and has been greatly advanced by deep neural networks (Lample et al., 2016; Chiu and Nichols, 2016; Peters et al., 2017). However, these ap- proaches generally require massive manually la- beled data, which prohibits their adaptation to low- resource languages due to high annotation costs. One solution to tackle that is to transfer knowl- edge from a source language with rich labeled data to a target language with little or even no labeled ℳ𝑟���𝑟𝑐 ℳ𝑟���𝑔𝑟��� Directly apply (i.e., ℳ𝑟���𝑔𝑟��� = ℳ𝑟���𝑟𝑐) {𝑋, 𝑋���}𝑟���𝑟𝑐 {𝑋′}𝑟���𝑔𝑟��� ℳ𝑟���𝑔𝑟��� Pairwise Relation ℳ𝑟���𝑟𝑐 {𝑋′}𝑟���𝑔𝑟��� ℳ𝑟���𝑔𝑟��� (b) (a) (c) {𝑋′, 𝑋���′}𝑟���𝑔𝑟��� {𝑋′, 𝑃′}𝑟���𝑔𝑟��� Training Training Figure 1: Comparison between previous cross-lingual NER methods (a/b) and the proposed method (c). (a): direct model transfer; (b): label projection with pairwise texts; (c): proposed teacher-student learn- ing method. Msrc/tgt: learned NER model for source/target language; {X, Y }src: labeled data in source language; {X′}tgt: unlabeled data in target lan- guage; {X′, Y ′}tgt/{X′, P ′}tgt: pseudo-labeled data in target language with hard labels / soft labels. data, which is referred to as cross-lingual NER (Wu and Dredze, 2019; Wu et al., 2020). In this paper, following Wu and Dredze (2019) and Wu et al. (2020), we focus on the extreme scenario of cross- lingual NER where no labeled data is available in the target language, which is …

Content truncated to 500 words