Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks

Proceedings of NAACL-HLT 2019, pages 3016–3025 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3016 Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks Ningyu Zhang1,2,3 Shumin Deng1,3 Zhanlin Sun1,3 Guanying Wang1,3 Xi Chen4 Wei Zhang2,3 Huajun Chen1,3∗ 1. College of Computer Science and Technology, Zhejiang University 2. Alibaba Group 3. AZFT†Joint Lab for Knowledge Engine 4. Tencent {3150105645,231sm,guanying wgy,huajunsir}@zju.edu.cn jasonxchen@tencent.com,{ningyu.zny,lantu.zw}@alibaba-inc.com Abstract We propose a distance supervised relation ex- traction approach for long-tailed, imbalanced data which is prevalent in real-world settings. Here, the challenge is to learn accurate ”few- shot” models for classes existing at the tail of the class distribution, for which little data is available. Inspired by the rich semantic cor- relations between classes at the long tail and those at the head, we take advantage of the knowledge from data-rich classes at the head of the distribution to boost the performance of the data-poor classes at the tail. First, we pro- pose to leverage implicit relational knowledge among class labels from knowledge graph em- beddings and learn explicit relational knowl- edge using graph convolution networks. Sec- ond, we integrate that relational knowledge into relation extraction model by coarse-to- fine knowledge-aware attention mechanism. We demonstrate our results for a large-scale benchmark dataset which show that our ap- proach significantly outperforms other base- lines, especially for long-tail relations. 1 Introduction Relation extraction (RE) is an important task in in- formation extraction, aiming to extract the relation between two given entities based on their related context. Due to the capability of extracting tex- tual information and benefiting many NLP appli- cations (e.g., information retrieval, dialog genera- tion, and question answering), RE appeals to many researchers. Conventional supervised models have been widely explored in this task (Zelenko et al., 2003; Zeng et al., 2014); however, their perfor- mance heavily depends on the scale and quality of training data. ∗ Corresponding author. †Alibaba-Zhejiang University Frontier Technology Re- search Center 0 10 20 30 40 50 Sorted Relation ID 0 10000 20000 30000 40000 50000 60000 70000 Relation Label Frequency ~40 relation labels occur <1000 times Figure 1: Label frequency distribution of classes with- out NA in NYT dataset. To construct large-scale data, (Mintz et al., 2009) proposed a novel distant supervision (DS) mechanism to automatically label training in- stances by aligning existing knowledge graphs (KGs) with text. DS enables RE models to work on large-scale training corpora and has thus be- come a primary approach for RE recently (Wu et al., 2017; Feng et al., 2018). Although these DS models achieve promising results on common relations, their performance still degrades dramat- ically when there are only a few training instances for some relations. Empirically, DS can automat- ically annotate adequate amounts of training data; however, this data usually only covers a limited part of the relations. Many relations are long-tail and still suffer from data deficiency. Current DS models ignore the problem of long-tail relations, which makes it challenging to extract comprehen- sive information from …

Content truncated to 500 words