Cooperative Denoising for Distantly Supervised Relation Extraction

Proceedings of the 27th International Conference on Computational Linguistics, pages 426–436 Santa Fe, New Mexico, USA, August 20-26, 2018. 426 Cooperative Denoising for Distantly Supervised Relation Extraction Kai Lei1,∗, Daoyuan Chen1,∗, Yaliang Li2, Nan Du2, Min Yang3, Wei Fan2, Ying Shen1,† 1School of Electronics and Computer Engineering, Peking University Shenzhen Graduate School 2Tencent Medical AI Lab 3Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences 1{leik, shenying}@pkusz.edu.cn, 1chendaoyuan@pku.edu.cn 2{yaliangli,ndu,davidwfan}@tencent.com, 3min.yang@siat.ac.cn Abstract Distantly supervised relation extraction greatly reduces human efforts in extracting relational facts from unstructured texts. However, it suffers from noisy labeling problem, which can de- grade its performance. Meanwhile, the useful information expressed in knowledge graph is still underutilized in the state-of-the-art methods for distantly supervised relation extraction. In the light of these challenges, we propose CORD, a novel COopeRative Denoising framework, which consists two base networks leveraging text corpus and knowledge graph respectively, and a co- operative module involving their mutual learning by the adaptive bi-directional knowledge distil- lation and dynamic ensemble with noisy-varying instances. Experimental results on a real-world dataset demonstrate that the proposed method reduces the noisy labels and achieves substantial improvement over the state-of-the-art methods. 1 Introduction Relation extraction aims to discover the semantic relationships between entities. Recently, it has at- tracted increasing attention due to its broad applications in many machine learning and natural language processing tasks such as Knowledge Graph (KG) construction (Shin et al., 2015), information retrieval (Kadry and Dietz, 2017), and question answering (Abujabal et al., 2017). Distant supervision is one of the most important techniques in practice for relation extraction due to its ability to generate large-scale labeled training data automatically by aligning KGs to text corpus. Despite its effectiveness, it suffers from noisy labeling problem such as false negative examples, which can severely degrade its performance. To alleviate this limitation, multi-instance learning and probabilistic graphical models have been widely explored by existing work (Riedel et al., 2010; Hoffmann et al., 2011; Surdeanu et al., 2012). Due to the success of deep learning, there has been increasing interest in applying deep neural networks to solve this problem. Zeng et al. (2015) proposed a piecewise CNN model combining multi-instance paradigm, Lin et al. (2016) and Lin et al. (2017) introduced sentence- level and multi-lingual attention to alleviate the side-effect caused by noisy instances, and Luo et al. (2017) further modeled noises explicitly. However, most existing studies reduce noises by leveraging information within text corpus, ignoring the relational facts expressed in other information sources, such as KG triples or semi-structured tables. Leveraging various information sources simultaneously, which takes full advantage of diverse and sup- plementary information in different sources, is beneficial to reduce noisy labels for distant supervision. To be more specific, we transform each sentence into entity sequence based on KG information, which is helpful to locate critical entities and adjust word-based network when their predictions are not consistent. Moreover, by incorporating information from other sources, distantly supervised relation extraction methods can better handle “Not A relation” (NA) class, which is …

Content truncated to 500 words