Named Entity Recognition via Noise Aware Training Mechanism with Data Filter

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4791–4803 August 1–6, 2021. ©2021 Association for Computational Linguistics 4791 Named Entity Recognition via Noise Aware Training Mechanism with Data Filter Xiusheng Huang1,2, Yubo Chen1,2, Shun Wu1, Jun Zhao1,2, Yuantao Xie3 and Weijian Sun3 1National Laboratory of Pattern Recognition, Institute of Automation, CAS, Beijing, China 2School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China 3Huawei Technologies Co., Ltd, Shenzhen, China huangxiusheng2020@ia.ac.cn, {yubo.chen,shun.wu,jzhao}@nlpr.ia.ac.cn, {xieyuantao2,sunweijian}@huawei.com Abstract Named entity recognition (NER) is a funda- mental task in natural language processing, these is a long held belief that datasets benefit the model. However, not all the data help with generalization, and some samples may contain ambiguous entities or noisy labels. The ex- isting methods can not distinguish hard sam- ples from noisy samples well, and becomes particularly challenging in the case of over- fitting. This paper proposes a new method called Noise-Aware-with-Filter (NAF) to solve the issues from two sides. From the perspec- tive of the data, we design a Logit-Maximum- Difference (LMD) mechanism, which max- imizes the diversity between different sam- ples to help the model identify noisy samples. From the perspective of the model, we design an Incomplete-Trust (In-trust) loss function, which boosts LCRF with a robust Distrust- Cross-Entropy(DCE) term. Our proposed In- trust can effectively alleviate the overfitting caused by previous loss function. Experiments on six real-world Chinese and English NER datasets show that NAF outperforms the previ- ous methods, and which obtained the state-of- the-art(SOTA) results on the CoNLL2003 and CoNLL++ datasets. 1 Introduction Named entity recognition (NER) is a primary task and which identifies both types and spans in sentences. NER models are becoming more and more accurate in prediction tasks, the potential im- provement of existing architectures in real-world applications is often inherently limited by data qual- ity (Pleiss et al., 2020). However, not all sam- ples are completely correct in the NER datasets (Nooralahzadeh et al., 2019; Lange et al., 2019). Many real-world datasets generally contain sam- ples which are “weakly-labeled” (Derczynski et al., 2017; Peng and Dredze, 2015). Specifically, some Model Logit Matrix In-trust loss Model Training In-trust loss Training Data Sentence Label Noise data Cleaner data LMD Sentence Label Label Figure 1: An overview of Noise-Aware-with- Filter(NAF). In-trust loss prevents model from overfitting and helps model generates logit matrices, which enters LMD mechanism with labels to filter noise data and get a cleaner data for model training. datasets which are annotated based on distant su- pervision (Yang et al., 2018; Liang et al., 2020) contain more noise labels, and manual annotators, especially on crowdsourcing platform, are prone to making labeling mistakes. Meanwhile labeling a huge datasets is an expensive and fallible process. As the increase of training iteration epochs, the model will overfit noisy samples and hinder the generalization of the model (Pleiss et al., 2020). In NER task, it is impractical to get an absolutely clean dataset, and the existing datasets generally exist mislabeled samples (Flor et al., 2019) and ambiguous …

Content truncated to 500 words