Relabel the Noise: Joint Extraction of Entities and Relations via Cooperative Multiagents

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5940–5950 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 5940 Relabel the Noise: Joint Extraction of Entities and Relations via Cooperative Multiagents Daoyuan Chen1 Yaliang Li1 Kai Lei2 Ying Shen3 ∗ 1Alibaba Group 2School of Electronics and Computer Engineering, Peking University 3School of Intelligent Systems Engineering, Sun Yat-Sen University 1{daoyuanchen.cdy, yaliang.li}@alibaba-inc.com 2leik@pkusz.edu.cn, 3sheny76@mail.sysu.edu.cn Abstract Distant supervision based methods for entity and relation extraction have received increas- ing popularity due to the fact that these meth- ods require light human annotation efforts. In this paper, we consider the problem of shifted label distribution, which is caused by the in- consistency between the noisy-labeled train- ing set subject to external knowledge graph and the human-annotated test set, and exacer- bated by the pipelined entity-then-relation ex- traction manner with noise propagation. We propose a joint extraction approach to address this problem by re-labeling noisy instances with a group of cooperative multiagents. To handle noisy instances in a fine-grained man- ner, each agent in the cooperative group eval- uates the instance by calculating a continuous confidence score from its own perspective; To leverage the correlations between these two ex- traction tasks, a confidence consensus module is designed to gather the wisdom of all agents and re-distribute the noisy training set with confidence-scored labels. Further, the confi- dences are used to adjust the training losses of extractors. Experimental results on two real- world datasets verify the benefits of re-labeling noisy instance, and show that the proposed model significantly outperforms the state-of- the-art entity and relation extraction methods. 1 Introduction The extraction of entities and relations has long been recognized as an important task within natu- ral language processing, as it facilitates text under- standing. The goal of the extraction task is to iden- tify entity mentions, assign predefined entity types, and extract their semantic relations from text cor- pora. For example, given a sentence “Washington is the president of the United States of America”, ∗Corresponding author. an extraction system will find a PRESIDENT OF re- lation between PERSON entity “Washington” and COUNTRY entity “United States of America”. A major challenge of the entity and relation extraction task is the absence of large-scale and domain-specific labeled training data due to the ex- pensive labeling efforts. One promising solution to address this challenge is distant supervision (DS) (Mintz et al., 2009; Hoffmann et al., 2011), which generates labeled training data automatically by aligning external knowledge graph (KG) to text corpus. Despite its effectiveness, the aligning pro- cess introduces many noisy labels that degrade the performance of extractors. To alleviate the intro- duced noise issue of DS, extensive studies have been performed, such as using probabilistic graphi- cal models (Surdeanu et al., 2012), neural networks with attention (Zeng et al., 2015; Lin et al., 2016) and instance selector with reinforcement learning (RL) (Qin et al., 2018; Feng et al., 2018). However, most existing works overlooked the shifted label distribution problem (Ye et al., …

Content truncated to 500 words