DIAG-NRE: A Neural Pattern Diagnosis Framework for Distantly Supervised Neural Relation Extraction

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1419–1429 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 1419 DIAG-NRE: A Neural Pattern Diagnosis Framework for Distantly Supervised Neural Relation Extraction Shun Zheng1 Xu Han2 Yankai Lin2 Peilin Yu3 Lu Chen1 Ling Huang1,4 Zhiyuan Liu2 Wei Xu1 1 Institute for Interdisciplinary Information Sciences, Tsinghua University, Beijing, China 2 Department of Computer Science and Technology, Tsinghua University, Beijing, China 3 Department of Computer Sciences, University of Wisconsin-Madison, Madison, USA 4 AHI Fintech Inc., Beijing, China {zhengs14,hanxu17,linyk14,lchen17}@mails.tsinghua.edu.cn; peilin@cs.wisc.edu; linghuang@fintec.ai; {liuzy,weixu}@tsinghua.edu.cn; Abstract Pattern-based labeling methods have achieved promising results in alleviating the inevitable labeling noises of distantly supervised neu- ral relation extraction. However, these meth- ods require significant expert labor to write relation-specific patterns, which makes them too sophisticated to generalize quickly. To ease the labor-intensive workload of pattern writing and enable the quick generalization to new relation types, we propose a neural pat- tern diagnosis framework, DIAG-NRE, that can automatically summarize and refine high- quality relational patterns from noise data with human experts in the loop. To demonstrate the effectiveness of DIAG-NRE, we apply it to two real-world datasets and present both sig- nificant and interpretable improvements over state-of-the-art methods. Source codes and data can be found at https://github. com/thunlp/DIAG-NRE. 1 Introduction Relation extraction aims to extract relational facts from the plain text and can benefit downstream knowledge-driven applications. A relational fact is defined as a relation between a head entity and a tail entity, e.g., (Letizia Moratti, Birthplace, Milan). The conventional methods often regard relation extraction as a supervised classification task that predicts the relation type between two de- tected entities mentioned in a sentence, including both statistical models (Zelenko et al., 2003; Zhou et al., 2005) and neural models (Zeng et al., 2014; dos Santos et al., 2015). These supervised models require a large number of human-annotated data to train, which are both expensive and time-consuming to collect. There- fore, Craven et al. (1999); Mintz et al. (2009) pro- posed distant supervision (DS) to automatically generate large-scale training data for relation ex- Knowledge Base Head Entity Tail Entity Relation Letizia Moratti Milan Birthplace Training Data for “Birthplace” Relation Sentence DS Label Ground Truth Error Type Marjorie_Kellogg was born in Santa_Barbara . 0 1 FN Mayor Letizia_Moratti of Milan disdainfully dismissed it . 1 0 FP Distant Supervision (DS) Figure 1: Two types of error labels, false negatives (FN) and false positives (FP), caused by DS. traction, by aligning relational facts from a knowl- edge base (KB) to plain text and assuming that every sentence mentioning two entities can de- scribe their relationships in the KB. As DS can ac- quire large-scale data without human annotation, it has been widely adopted by recent neural rela- tion extraction (NRE) models (Zeng et al., 2015; Lin et al., 2016). Although DS is both simple and effective in many cases, it inevitably introduces intolerable la- beling noises. As Figure 1 shows, there are two types …

Content truncated to 500 words