MulDA: A Multilingual Data Augmentation Framework for Low-Resource Cross-Lingual NER

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 5834–5846 August 1–6, 2021. ©2021 Association for Computational Linguistics 5834 MulDA: A Multilingual Data Augmentation Framework for Low-Resource Cross-Lingual NER Linlin Liu∗12 Bosheng Ding∗12 Lidong Bing2 Shafiq Joty1 Luo Si2 Chunyan Miao1 1Nanyang Technological University, Singapore 2DAMO Academy, Alibaba Group {linlin.liu, bosheng.ding, l.bing, luo.si}@alibaba-inc.com {srjoty, ascymiao}@ntu.edu.sg Abstract Named Entity Recognition (NER) for low- resource languages is a both practical and challenging research problem. This paper addresses zero-shot transfer for cross-lingual NER, especially when the amount of source- language training data is also limited. The paper first proposes a simple but effective la- beled sequence translation method to trans- late source-language training data to target languages and avoids problems such as word order change and entity span determination. With the source-language data as well as the translated data, a generation-based multilin- gual data augmentation method is introduced to further increase diversity by generating synthetic labeled data in multiple languages. These augmented data enable the language model based NER models to generalize bet- ter with both the language-specific features from the target-language synthetic data and the language-independent features from multilin- gual synthetic data. An extensive set of ex- periments were conducted to demonstrate en- couraging cross-lingual transfer performance of the new research on a wide variety of target languages.1 1 Introduction Named entity recognition (NER) aims to identify and classify entities in a text into predefined types, which is an essential tool for information extraction. It has also been proven to be useful in various down- stream natural language processing (NLP) tasks, including information retrieval (Banerjee et al., 2019), question answering (Fabbri et al., 2020) and text summarization (Nallapati et al., 2016). However, except for some resource-rich languages ∗Equal contribution, order decided by coin flip. Linlin Liu and Bosheng Ding are under the Joint PhD Program between Alibaba and Nanyang Technological University. 1Our code is available at https://ntunlpsg. github.io/project/mulda/. (e.g., English, German), training sets for most of the other languages are still very limited. More- over, it is usually expensive and time-consuming to annotate such data, particularly for low-resource languages (Kruengkrai et al., 2020). Therefore, zero-shot cross-lingual NER has attracted growing interest recently, especially with the influx of deep learning methods (Mayhew et al., 2017; Joty et al., 2017; Jain et al., 2019; Bari et al., 2021). Existing approaches to cross-lingual NER can be roughly grouped into two main categories: instance-based transfer via machine translation (MT) and label projection (Mayhew et al., 2017; Jain et al., 2019), and model-based transfer with aligned cross-lingual word representations or pre- trained multilingual language models (Joty et al., 2017; Baumann, 2019; Wang et al., 2020; Conneau et al., 2020; Bari et al., 2021). Recently, Wu et al. (2020) unify instance-based and model-based trans- fer via knowledge distillation. These recent methods have demonstrated promis- ing zero-shot cross-lingual NER performance. However, most of them assume the availability of a considerable amount of training data in …

Content truncated to 500 words