H-FND: Hierarchical False-Negative Denoising for Distant Supervision Relation Extraction

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2579–2593 August 1–6, 2021. ©2021 Association for Computational Linguistics 2579 H-FND: Hierarchical False-Negative Denoising for Distant Supervision Relation Extraction Jhih-Wei Chen*1, Tsu-Jui Fu*2, Chen-Kang Lee3, and Wei-Yun Ma3 1University of California Los Angeles 2University of California Santa Barbara 3Institute of Information Science, Academia Sinica jwchen101@g.ucla.com, tsu-juifu@ucsb.edu {cklee, ma}@iis.sinica.edu.tw Abstract Although distant supervision automatically generates training data for relation extraction, it also introduces false-positive (FP) and false- negative (FN) training instances to the gener- ated datasets. While both types of errors de- grade the final model performance, previous work on distant supervision denoising focuses more on suppressing FP noise and less on re- solving the FN problem. We here propose H-FND, a hierarchical false-negative denois- ing framework for robust distant supervision relation extraction, as an FN denoising solu- tion. H-FND uses a hierarchical policy which first determines whether non-relation (NA) in- stances should be kept, discarded, or revised during the training process. For those learning instances which are to be revised, the policy further reassigns them appropriate relations, making them better training inputs. Experi- ments on SemEval-2010 and TACRED were conducted with controlled FN ratios that ran- domly turn the relations of training and vali- dation instances into negatives to generate FN instances. In this setting, H-FND can revise FN instances correctly and maintains high F1 scores even when 50% of the instances have been turned into negatives. Experiments on NYT10 is further conducted to show that H- FND is applicable in a realistic setting. 1 1 Introduction Relation extraction (Zelenko et al., 2003; Mooney and Bunescu, 2006; Zhou et al., 2005) is a core task in information extraction. Its goal is to deter- mine the relation between two entities in a given sentence. For instance, given the sentence “Jobs was born in San Francisco”, with head and tail en- tities “Jobs” and “San Francisco”, the relation to be extracted is “Place of Birth”. Relation extrac- tion can be applied for many applications, such *Equal contribution. 1The code can be found at https://github.com/ ckiplab/hfnd Knowledge base Relation Steve Jobs, San Francisco PoB Corpus Relation Type Jobs was born in San Francisco PoB () TP Jobs moved back to San Francisco PoB () FP Manuela was born in New York NA () FN Table 1: Distant supervision and different types of in- correctly labeled relations. The head and tail entities are shown in boldface, and “PoB” stands for the rela- tion “Place of Birth”. as question answering and knowledge graph com- pletion. A major difficulty with supervising re- lation extraction models is the cost of collect- ing training data, against which distant supervi- sion (DS) (Hoffmann et al., 2011; Surdeanu et al., 2012) is proposed. DS obtains the relational facts from a knowledge base and aligns these facts to all sentences in the corpus to generate learning instances. In specific, if a relation triple r(h, t) exists in a knowledge base, then for a sentence s which mentions both the head entity …

Content truncated to 500 words