Connecting Language and Knowledge with Heterogeneous Representations for Neural Relation Extraction

Proceedings of NAACL-HLT 2019, pages 3201–3206 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3201 Connecting Language and Knowledge with Heterogeneous Representations for Neural Relation Extraction Peng Xu Department of Computing Science University of Alberta Edmonton, Canada pxu4@ualberta.ca Denilson Barbosa Department of Computing Science University of Alberta Edmonton, Canada denilson@ualberta.ca Abstract Knowledge Bases (KBs) require constant up- dating to reflect changes to the world they rep- resent. For general purpose KBs, this is of- ten done through Relation Extraction (RE), the task of predicting KB relations expressed in text mentioning entities known to the KB. One way to improve RE is to use KB Embeddings (KBE) for link prediction. However, despite clear connections between RE and KBE, lit- tle has been done toward properly unifying these models systematically. We help close the gap with a framework that unifies the learning of RE and KBE models leading to significant improvements over the state-of-the-art in RE. The code is available at https://github. com/billy-inn/HRERE. 1 Introduction Knowledge Bases (KBs) contain structured infor- mation about the world and are used in support of many natural language processing applications such as semantic search and question answering. Building KBs is a never-ending challenge because, as the world changes, new knowledge needs to be harvested while old knowledge needs to be re- vised. This motivates the work on the Relation Ex- traction (RE) task, whose goal is to assign a KB relation to a phrase connecting a pair of entities, which in turn can be used for updating the KB. The state-of-the-art in RE builds on neural mod- els using distant (a.k.a. weak) supervision (Mintz et al., 2009) on large-scale corpora for training. A task related to RE is that of Knowledge Base Embedding (KBE), which is concerned with rep- resenting KB entities and relations in a vector space for predicting missing links in the graph. Aiming to leverage the similarities between these tasks, Weston et al. (2013) were the first to show that combining predictions from RE and KBE models was beneficial for RE. However, the way in which they combine RE and KBE predictions is rather naive (namely, by adding those scores). To the best of our knowledge, there have been no systematic attempts to further unify RE and KBE, particularly during model training. We seek to close this gap with HRERE (Het- erogeneous REpresentations for neural Relation Extraction), a novel neural RE framework that learns language and knowledge representations jointly. Figure 1 gives an overview. HRERE’s backbone is a bi-directional long short term mem- ory (LSTM) network with multiple levels of atten- tion to learn representations of text expressing re- lations. The knowledge representation machinery, borrowed from ComplEx (Trouillon et al., 2016), nudges the language model to agree with facts in the KB. Joint learning is guided by three loss func- tions: one for the language representation, another for the knowledge representation, and a third one to ensure these representations do not diverge. In effect, this contributes to HRERE’s …

Content truncated to 500 words