A Soft-label Method for Noise-tolerant Distantly Supervised Relation Extraction

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1790–1795 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics A Soft-label Method for Noise-tolerant Distantly Supervised Relation Extraction Tianyu Liu, Kexiang Wang, Baobao Chang and Zhifang Sui Key Laboratory of Computational Linguistics, Ministry of Education, School of Electronics Engineering and Computer Science, Peking University, Beijing, China {tianyu0421, wkx, chbb, szf}@pku.edu.cn Abstract Distant-supervised relation extraction in- evitably suffers from wrong labeling prob- lems because it heuristically labels rela- tional facts with knowledge bases. Pre- vious sentence level denoise models don’t achieve satisfying performances because they use hard labels which are determined by distant supervision and immutable dur- ing training. To this end, we introduce an entity-pair level denoise method which ex- ploits semantic information from correctly labeled entity pairs to correct wrong labels dynamically during training. We propose a joint score function which combines the relational scores based on the entity-pair representation and the confidence of the hard label to obtain a new label, namely a soft label, for certain entity pair. During training, soft labels instead of hard labels serve as gold labels. Experiments on the benchmark dataset show that our method dramatically reduces noisy instances and outperforms the state-of-the-art systems. 1 Introduction Relation Extraction (RE) aims to obtain relational facts from plain text. Traditional supervised RE systems suffer from lack of manually labeled data. Mintz et al. (2009) proposes distant supervision, which exploits relational facts in knowledge bases (KBs). Distant supervision automatically gener- ates training examples by aligning entity mentions in plain text with those in KB and labeling entity pairs with their relations in KB. If there’s no re- lation link between certain entity pair in KB, it will be labeled as negative instance (NA). How- ever, the automatic labeling inevitably accompa- nies with wrong labels because the relations of !"#$ %&'() *+,-./ !012.31)4$&5(6/ !789: ;(:) 78(:&/ !;&: *6(&221:)2<9=9:/ >? @: &:(:.9$A(9< !"# $%&"'( ')# 1BC'*+,+# 2&(=D E- !"# $%&"'')#) <81 (2 '*+( ,+# F2 &#G&22&=1$C2&(=D H? !"# $%&"'')#1BC'*+,+#) .89C-$92(=9:.C1B D >? I(:C./"#0+ )C<9C&6<&,2C(#J &+(:9 A(169:K9C???L2&(=1+2"'( 3&"#45)06) &C-16(.(K&6 2K(9:.(J 2. <81C2-9K(&6(592C(:D !M9G&2.(&:CN1K8) O$&:K9/ !@=( "#(:) P+&:=&/ >? 7,& 89&#41BC:;"#," =(9=C(:C2&Q=( &$&G(& ? E? D &B.9$C.89C=(K.&.1$ 7,& 89&#41BC<;"#," 9R-J 9669=.89#C&:=C1.89$ 9.8:(KC&2(&:2 B$1#C.89 K1Q:.$,? !"#$ %&'()(*$ S&.(1:&6(.,+7,& 89&#, :;"#,"- S&.(1:&6(.,+=6+#!&#, =6&#"- S&.(1:&6(.,+>)'3?), @/"A&%- S&.(1:&6(.,+89/B"C&, $;DE3- O&629 S9+&.(A9 S" !!"# $%&"'')#F1*+,+#/ ./)(&0/1()2 O&629 T12(.(A9 S&.(1:&6(.,!1+2"'3&"#45)06) ./"#0+/ .3 DD D O&629 S9+&.(A9 O&629 T12(.(A9 0$Q9 T12(.(A9 Figure 1: An example of soft-label correction on Nationality relation. We intend to use syntactic/ semantic information of correctly labeled entity pairs (blue) to correct the false positive and false negative instances (orange) during training. entity pairs might be missing from KBs or mis- labeled. Multi-instances learning (MIL) is proposed by Riedel et al. (2010) to combat the noise. The method divides the training set into multiple bags of entity pairs (shown in Fig 1) and labels the bags with the relations of entity pairs in the KB. Each bag consists of sentences mentioning both head and tail entities. Much effort …

Content truncated to 500 words