Exploiting Noisy Data in Distant Supervision Relation Classification

Proceedings of NAACL-HLT 2019, pages 3216–3225 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3216 Exploiting Noisy Data in Distant Supervision Relation Classification Kaijia Yang, Liang He, Xin-yu Dai, Shujian Huang, Jiajun Chen National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210023, China {yangkj,heliang}@nlp.nju.edu.cn {daixinyu,huangsj,chenjj}@nju.edu.cn Abstract Distant supervision has obtained great progress on relation classification task. However, it still suffers from noisy labeling problem. Different from previous works that underutilize noisy data which inherently characterize the property of classification, in this paper, we propose RCEND, a novel framework to enhance Relation Classification by Exploiting Noisy Data. First, an instance discriminator with reinforcement learning is designed to split the noisy data into correctly labeled data and incorrectly labeled data. Second, we learn a robust relation classifier in semi-supervised learning way, whereby the correctly and incorrectly labeled data are treated as labeled and unlabeled data respec- tively. The experimental results show that our method outperforms the state-of-the-art models. 1 Introduction Relation classification plays a crucial role in natu- ral language processing (NLP) tasks, such as ques- tion answering and knowledge base completion (Xu et al., 2016; Han et al., 2018a). The goal of relation classification is to predict relations of the target entity pair given a plain text. Traditional su- pervised learning methods (Zelenko et al., 2002; Bunescu and Mooney, 2005; Zhou et al., 2005) heavily rely on large scale annotated data which is time and labor consuming. Mintz et al. (2009) pro- posed distant supervision (DS) to automatically generate training data for relation classification based on the assumption that if two target entities have a relation in knowledge base (KB), sentences containing this entity pair might express the re- lation. For example, if a relational fact <Apple, founder, Steve Jobs> exists in KB, distant super- vision will assign founder as the label of all sen- tences that contain “Apple” and “Steve Jobs” to- gether. Sentence DS Gold S1:Al Gore was waiting to board a commercial flight from Nashville to Miami... LivedIn NA S2:There were also performers who were born in Louisiana , in- cluding Lucinda Williams... LivedIn BornIn S3:Boggs was married, had three young children and lived in Brew- ster NA LivedIn Table 1: Examples of noisy labeling problem in dis- tant supervision relation classification. S1 and S2 are heuristically labeled as LivedIn by DS, but neither of them mention the relation while S2 mentions the BornIn relation. S3 expresses the LivedIn relation but it is mislabeled as NA since no relation of the entity pair exist in KB. However, it suffers from noisy labeling problem due to the irrelevance of aligned text and incom- pleteness of KB, which consists of false positives and false negatives. The false positives means that not all sentences containing two entities mention the relation in KB, such as S1 and S2 in Table 1. And the false negatives are sentences are misla- beled as no relation (NA) due to the absence of relational fact in KB even though …

Content truncated to 500 words