Improving Low-Resource Named Entity Recognition via Label-Aware Data Augmentation and Curriculum Denoising

Improving Low-Resource Named Entity Recognition via Label-Aware Data Augmentation and Curriculum Denoising Wenjing Zhu∗, Jian Liu∗, Jinan Xu†, Yufeng Chen, Yujie Zhang School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China {18120461,jianliu,jaxu,chenyf,yjzhang}@bjtu.edu.cn Abstract Deep neural networks have achieved state-of-the-art performances on named entity recognition (NER) with sufficient training data, while they perform poorly in low-resource scenarios due to data scarcity. To solve this problem, we propose a novel data augmentation method based on pre- trained language model (PLM) and curriculum learning strategy. Concretely, we use the PLM to generate diverse training instances through predicting different masked words and design a task-specific curriculum learning strategy to alleviate the influence of noises. We evaluate the effectiveness of our approach on three datasets: CoNLL-2003, OntoNotes5.0, and MaScip, of which the first two are simulated low-resource scenarios, and the last one is a real low-resource dataset in material science domain. Experimental results show that our method consistently outperform the baseline model. Specifically, our method achieves an absolute improvement of 3.46% F1 score on the 1% CoNLL-2003, 2.58% on the 1% OntoNotes5.0, and 0.99% on the full of MaScip. 1 Introduction Named entity recognition (NER) is a fundamental natural language processing (NLP) task aiming to identify the names of people, places, organizations, and proper nouns in texts, which supports a wide range of downstream applications (Huang et al., 2015; Kuru et al., 2016). The current state-of-the-art methods for NER rely on abundant training data. However, manual annotation is expensive, which limits the effectiveness of the model, especially in bio-medicine and material chemistry domains (Friedrich et al., 2020). Many studies have investigated NER in low-resource scenarios, by transferring pre-trained language representations on self-supervised or rich-resource domains to target domains (Ruder, 2019; Gururangan et al., 2020). Others use the knowledge base to semi-automatically label extra data for training (Zeng et al., 2015). Nevertheless, these methods usually require huge expertise knowledge to obtain good performance. Data augmentation has been proven effective to alleviate data scarcity in many NLP tasks, including machine translation (Wang et al., 2018; Gao et al., 2019), text classification (Wei and Zou, 2019; Xie et al., 2020), question answering (Raiman and Miller, 2017), etc. (Min et al., 2020). However, most existing studies focus on sentence-level tasks, which generate sentences via word replacement, swap, and deletion (Wei and Zou, 2019; Min et al., 2020) or generative models (Yu et al., 2018; Iyyer et al., 2018). Different from these sentence-level NLP tasks, NER predict entities on the token level. That is, for each token in the sentence, NER models predict a label indicating whether the token belongs to a mention and which entity type the mention has. Therefore, applying transformations to tokens may also change their labels. Due to this difficulty, data augmentation for NER is comparatively less studied. In this work, we propose a novel data augmentation framework for NER in low-resource scenarios, which generates examples with consistent labels and filters noises in the generated data. Concretely, our approach contains two complementary components: …

Content truncated to 500 words