Improving Distantly-Supervised Relation Extraction with Joint Label Embedding

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3821–3829, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3821 Improving Distantly-Supervised Relation Extraction with Joint Label Embedding Linmei Hu1, Luhao Zhang1, Chuan Shi1 ∗, Liqiang Nie2, Weili Guan3, Cheng Yang1 1Beijing University of Posts and Telecommunications, China 2Shan Dong University, China 3Hewlett Packard Enterprise Singapore, Singapore {hulinmei,zhangluhao,shichuan}@bupt.edu.cn {nieliqiang, honeyguan, albertyang33}@gmail.com Abstract Distantly-supervised relation extraction has proven to be effective to find relational facts from texts. However, the existing approaches treat labels as independent and meaningless one-hot vectors, which cause a loss of poten- tial label information for selecting valid in- stances. In this paper, we propose a novel multi-layer attention-based model to improve relation extraction with joint label embedding. The model makes full use of both structural information from Knowledge Graphs and tex- tual information from entity descriptions to learn label embeddings through gating integra- tion, while avoiding the imposed noise with an attention mechanism. Then the learned label embeddings are used as another atten- tion over the instances (whose embeddings are also enhanced with the entity descriptions) for improving relation extraction. Extensive ex- periments demonstrate that our model signifi- cantly outperforms state-of-the-art methods. 1 Introduction Knowledge Graphs (KGs) such as Freebase and DBpedia have shown their strong power in many natural language processing tasks including ques- tion answering (Zhang et al., 2018) and dialog generation (Zhou et al., 2018). However, these KGs are far from complete. Relation extraction, which aims to fill this gap by extracting semantic relationships between entity pairs from plain texts, is thus of great importance. Most existing supervised relation extraction methods require a large number of labeled train- ing data, which is time-consuming and laborious. Distant supervision has been proposed by (Mintz et al., 2009) to address the challenge. It assumes that if two entities have a relation in KGs, then all sentences mentioning the two entities express ∗Corresponding author: Chuan Shi. Romania … Its capital and largest city is Bucharest … include Cluj-Napoca… Knowledge Graph Entity Description Entity Description A bag of sentences Method (2) (a) Existing Methods (b) Our Proposed Method Label Embedding Classifier Att Att Classifier Method (3) Knowledge Graph A bag of sentences Att Textual Relation Embedding Classifier Method (1) Att Textual Relation Embedding Figure 1: Illustration of comparison of existing meth- ods and our proposed method. this relation. Thus, distant supervision can auto- matically generate a large number of labeled data without labor cost. Simultaneously, it often suf- fers from wrong labeling problem (Surdeanu et al., 2012; Zeng et al., 2015). Recently, significant progress has been made in applying deep neural networks for relation extrac- tion under distant supervision (Zeng et al., 2014, 2015; Feng et al., 2017). To alleviate the wrong labeling problem in distant supervision, attention models have been proposed to select valid in- stances (Ji et al., 2017). As shown in Figure 1 (a), they can be divided into three …

Content truncated to 500 words