Learning from Noisy Labels for Entity-Centric Information Extraction

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5381–5392 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5381 Learning from Noisy Labels for Entity-Centric Information Extraction Wenxuan Zhou University of Southern California zhouwenx@usc.edu Muhao Chen University of Southern California muhaoche@usc.edu Abstract Recent information extraction approaches have relied on training deep neural models. However, such models can easily overfit noisy labels and suffer from performance degrada- tion. While it is very costly to filter noisy la- bels in large learning resources, recent studies show that such labels take more training steps to be memorized and are more frequently for- gotten than clean labels, therefore are identifi- able in training. Motivated by such properties, we propose a simple co-regularization frame- work for entity-centric information extraction, which consists of several neural models with identical structures but different parameter ini- tialization. These models are jointly optimized with the task-specific losses and are regular- ized to generate similar predictions based on an agreement loss, which prevents overfitting on noisy labels. Extensive experiments on two widely used but noisy benchmarks for infor- mation extraction, TACRED and CoNLL03, demonstrate the effectiveness of our frame- work. We release our code to the community for future research1. 1 Introduction Deep neural models have achieved significant suc- cess on various information extraction (IE) tasks. However, when training labels contain noise, deep neural models can easily overfit the noisy labels, leading to severe performance degradation (Arpit et al., 2017; Zhang et al., 2017a). Unfortunately, labeling on large corpora, regardless of using hu- man annotation (Raykar et al., 2010) or automated heuristics (Song et al., 2015), inevitably suffers from labeling errors. This problem has even dras- tically affected widely used benchmarks, such as CoNLL03 (Sang and De Meulder, 2003) and TA- CRED (Zhang et al., 2017b), where a notable por- tion of incorrect labels have been caused in annota- tion and largely hindered the performance of SOTA 1Our code is publically available at https://github. com/wzhouad/NLL-IE <latexit sha1_base64="l04wbJfy/oET64onEaUZ07RrXY=">AB7nicbVBNSwMxEJ31s9avqkcvwSLUS9mIqMeiF48V7Ae0a 8m2TY0mw1JVihLf4QXD4p49fd489+YtnvQ1gcDj/dmJkXKsGN9f1vb2V1bX1js7BV3N7Z3dsvHRw2TZJqyho0EYluh8QwSVrWG4FayvNSBwK1gpHt1O/9cS04Yl8sGPFgpgMJI84JdZJLfWYVfDZpFcq+1V/BrRMcE7KkKPeK31+wlNYyYtFcSYDvaVDTKiLaeCT Yrd1DBF6IgMWMdRSWJmgmx27gSdOqWPokS7khbN1N8TGYmNGceh64yJHZpFbyr+53VSG10HGZcqtUzS+aIoFcgmaPo76nPNqBVjRwjV3N2K6JBoQq1LqOhCwIsvL5PmeRVfVvH9Rbl2k8dRgGM4gQpguIa3EdGkBhBM/wCm+e8l68d+9j3ri5TNH8Afe5w+MiI8O< /latexit> p(1) <latexit sha1_base64="maY5H75yoLbHAJO0Qagak9JdQn8=">AB6HicbVBNS8NAEJ34WetX1a OXxSJ4KomIeix68diC/YA2lM120q7dbOLuRihv8CLB0W8+pO8+W/ctjlo64OBx3szMwLEsG1cd1vZ2V1bX1js7BV3N7Z3dsvHRw2dZwqhg0Wi1i1A6pRcIkNw43AdqKQRoHAVjC6nfqtJ1Sax/LejBP0I zqQPOSMGivVH3ulsltxZyDLxMtJGXLUeqWvbj9maYTSMEG17nhuYvyMKsOZwEmxm2pMKBvRAXYslTRC7WezQyfk1Cp9EsbKljRkpv6eyGik9TgKbGdEzVAvelPxP6+TmvDaz7hMUoOSzReFqSAmJtOvSZ8r ZEaMLaFMcXsrYUOqKDM2m6INwVt8eZk0zyveZcWrX5SrN3kcBTiGEzgD6gCndQgwYwQHiGV3hzHpwX5935mLeuOPnMEfyB8/kD3ZWM+w=</latexit>q <latexit sha1_base64="PnAQV+E8GuM12aj79V0t8KsJS+4=">AB6HicbVBNS8NAEJ3Ur1 q/qh69LBbBU0lE1GPRi8cW7Ae0oWy2k3btZhN2N0Io/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZLGLVCahGwSU2DTcCO4lCGgUC28H4bua3 n1BpHsHkyXoR3QoecgZNVZqZP1yxa26c5BV4uWkAjnq/fJXbxCzNEJpmKBadz03Mf6EKsOZwGmpl2pMKBvTIXYtlTRC7U/mh07JmVUGJIyVLWnIXP09MaGR1lkU2M6ImpFe9mbif143NeGNP+E ySQ1KtlgUpoKYmMy+JgOukBmRWUKZ4vZWwkZUWZsNiUbgrf8ipXVS9q6rXuKzUbvM4inACp3AOHlxDe6hDk1gPAMr/DmPDovzrvzsWgtOPnMfyB8/kD6bWNAw=</latexit>y <latexit sha1_base64="A3prOmZyL0TnlFarzY9NizXqrE=">AB7nicbVBNSwMxEJ2tX7V+VT16CRahXsquiHosevEiVLAf0 K4lm2b0Gw2JFmhLPsjvHhQxKu/x5v/xrTdg7Y+GHi8N8PMvEBypo3rfjuFldW19Y3iZmlre2d3r7x/0NJxoghtkpjHqhNgTkTtGmY4bQjFcVRwGk7GN9M/fYTVZrF4sFMJPUjPBQsZAQbK7XlY1q9O8365Ypbc2dAy8TLSQVyNPrlr94gJklEhSEca931XGn8FCvDC KdZqZdoKjEZ4yHtWipwRLWfzs7N0IlVBiMlS1h0Ez9PZHiSOtJFNjOCJuRXvSm4n9eNzHhlZ8yIRNDBZkvChOTIymv6MBU5QYPrE8XsrYiMsMLE2IRKNgRv8eVl0jqreRc17/68Ur/O4yjCERxDFTy4hDrcQgOaQGAMz/AKb450Xpx352PeWnDymUP4A+fzB7cwj yo=</latexit> p(M) <latexit sha1_base64="nZAf4qy5S6Xv1YlpZ4mJ7+47hk=">AB7nicbVBNSwMxEJ2tX7V+VT16CRahXspuEfVY9OKxgv2Ad i3ZNuGZrMhyQpl6Y/w4kERr/4eb/4bs+0etPXBwO9GWbmBZIzbVz32ymsrW9sbhW3Szu7e/sH5cOjto4TRWiLxDxW3QBrypmgLcMp12pKI4CTjvB5DbzO09UaRaLBzOV1I/wSLCQEWys1JGPabV+PhuUK27NnQOtEi8nFcjRHJS/+sOYJBEVhnCsdc9zpfFTrAwjn M5K/URTickEj2jPUoEjqv10fu4MnVliMJY2RIGzdXfEymOtJ5Gge2MsBnrZS8T/N6iQmv/ZQJmRgqyGJRmHBkYpT9joZMUWL41BJMFLO3IjLGChNjEyrZELzl1dJu17zLmve/UWlcZPHUYQTOIUqeHAFDbiDJrSAwASe4RXeHOm8O/Ox6K14OQzx/AHzucPjg6PD w=</latexit> p(2) model 1 model 2 model M … … aggregation of p(1) to p(M) ground-truth labels back-propagation mean aggregation Figure 1: Illustration of our co-regularization frame- work. The base models are jointly optimized with the task-specific loss from label y and an agreement loss, which regularizes the models to generate similar pre- dictions to the aggregated soft target probability q. systems (Reiss et al., 2020; Alt et al., 2020). Hence, developing a robust learning method that better tol- erates noisy supervision represents an urged chal- lenge for emerging IE models. So far, few research efforts have been made to developing noise-robust IE models, and existing work mainly focuses on the weakly supervised or distantly supervised setting (Surdeanu et al., 2012; Ratner et al., 2016; Huang and Du, 2019; Mayhew et al., 2019). Most of such methods typically de- pend on multi-instance learning that relies on …

Content truncated to 500 words