CoLaDa: A Collaborative Label Denoising Framework for Cross-lingual Named Entity Recognition

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 5995–6009 July 9-14, 2023 ©2023 Association for Computational Linguistics CoLaDa: A Collaborative Label Denoising Framework for Cross-lingual Named Entity Recognition Tingting Ma1∗, Qianhui Wu2, Huiqiang Jiang2, Börje F. Karlsson2, Tiejun Zhao1†, Chin-Yew Lin2 1Harbin Institute of Technology, Harbin, China 2Microsoft hittingtingma@gmail.com {qianhuiwu, hjiang, borjekar, cyl}@microsoft.com tjzhao@hit.edu.cn Abstract Cross-lingual named entity recognition (NER) aims to train an NER system that generalizes well to a target language by leveraging labeled data in a given source language. Previous work alleviates the data scarcity problem by translat- ing source-language labeled data or performing knowledge distillation on target-language unla- beled data. However, these methods may suffer from label noise due to the automatic labeling process. In this paper, we propose CoLaDa, a Collaborative Label Denoising Framework, to address this problem. Specifically, we first explore a model-collaboration-based denoising scheme that enables models trained on different data sources to collaboratively denoise pseudo labels used by each other. We then present an instance-collaboration-based strategy that considers the label consistency of each token’s neighborhood in the representation space for denoising. Experiments on different bench- mark datasets show that the proposed CoLaDa achieves superior results compared to previous methods, especially when generalizing to dis- tant languages.1 1 Introduction The named entity recognition (NER) task aims to locate and classify entity spans in a given text into predefined entity types. It is widely used for many downstream applications, such as relation extrac- tion and question answering. Deep neural networks have made significant progress on this task lever- aging large-scale human-annotated data for train- ing. However, fine-grained token-level annotation makes it costly to collect enough high-quality la- beled data, especially for low-resource languages. Such scenarios motivate the research on zero-shot cross-lingual NER, which attempts to leverage la- beled data in a rich-resource source language to *Work during internship at Microsoft. †Corresponding author. 1Our code is available at https://github.com/ microsoft/vert-papers/tree/master/papers/CoLaDa. 𝒟𝑟���𝑟𝑐 𝒟𝑟���𝑟𝑎𝑛𝑟��� ℳ𝑟���𝑟𝑎𝑛𝑟��� Translate & Label projection Training (a) TMP 𝒟𝑟���𝑔𝑟��� ℳ𝑟���𝑔𝑟��� Training ℳ𝑟���𝑟𝑐 Soft Label (b) TSL ℳ𝑟���𝑔𝑟��� (𝑖) 𝒟𝑟���𝑔𝑟��� 𝒟𝑟���𝑟𝑎𝑛𝑟��� ℳ𝑟���𝑟𝑎𝑛𝑟��� (𝑖) Denoise Noise-robust Training Re-label Noise-robust Training (d) CoLaDa 𝒟𝑟���𝑔𝑟��� ℳ𝑟���𝑔𝑟��� Training ℳ𝑟���𝑟𝑎𝑛𝑟��� Soft Label (c) UniTrans Figure 1: Comparison between previous methods (a/b/c) and our CoLaDa at the i-th iteration (d) CoLaDa starts at M0 tgt and performs denoising iteratively. Dsrc: Source- language labeled data. Dtrans: Translation data. Dtgt: Target-language unlabeled data with pseudo-labels gen- erated by NER models. Msrc/trans/tgt: NER model learned on Dsrc/trans/tgt. solve the NER task in a target language without annotated data. Recent attempts at cross-lingual NER can be roughly categorized from two aspects: learning language-independent features via feature align- ment (Huang et al., 2019; Keung et al., 2019) and learning language-specific features from au- tomatically labeled target-language data (Wu et al., 2020c,b). Despite bringing great success to cross- lingual NER, the former line of research misses ex- ploiting language-specific features and thus shows substandard performance, especially when trans- ferring to distant languages, e.g., from English to Arabic …

Content truncated to 500 words