Learning Relational Representations by Analogy using Hierarchical Siamese Networks

Proceedings of NAACL-HLT 2019, pages 3235–3245 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3235 Learning Relational Representations by Analogy using Hierarchical Siamese Networks Gaetano Rossiello†, Alfio Gliozzo‡, Robert Farrell‡, Nicolas Fauceglia‡, Michael Glass‡ †Department of Computer Science, University of Bari, Italy ‡IBM Research AI, Yorktown Heights, NY, US gaetano.rossiello@uniba.it, gliozzo@us.ibm.com, robfarr@us.ibm.com, nicolas.fauceglia@ibm.com, mrglass@us.ibm.com Abstract We address relation extraction as an analogy problem by proposing a novel approach to learn representations of relations expressed by their textual mentions. In our assumption, if two pairs of entities belong to the same rela- tion, then those two pairs are analogous. Fol- lowing this idea, we collect a large set of anal- ogous pairs by matching triples in knowledge bases with web-scale corpora through distant supervision. We leverage this dataset to train a hierarchical siamese network in order to learn entity-entity embeddings which encode rela- tional information through the different lin- guistic paraphrasing expressing the same re- lation. We evaluate our model in a one-shot learning task by showing a promising gen- eralization capability in order to classify un- seen relation types, which makes this approach suitable to perform automatic knowledge base population with minimal supervision. More- over, the model can be used to generate pre- trained embeddings which provide a valuable signal when integrated into an existing neural- based model by outperforming the state-of- the-art methods on a downstream relation ex- traction task. 1 Introduction The task of identifying semantic relationships be- tween entities in unstructured textual corpora, namely Relation Extraction (RE), is often a pre- requisite for many other natural language under- standing tasks, e.g. automatic knowledge base population, question answering, etc. RE is com- monly addressed as a classification task (Bunescu et al., 2005), where a model is trained to classify relation mentions in text among a predefined set of relation types. For instance, given the sentence “Robert Plant is the singer of the band Led Zep- pelin”, an effective RE system might extract the triple memberOf(ROBERT PLANT, LED ZEP- PELIN), where memberOf is a relation label ex- pressed by the linguistic context “is the singer of the band”. Since a given relation can be expressed us- ing different textual patterns surrounding entities, the state-of-the-art RE models which follow this approach need a considerable amount of exam- ples for each relation to reach satisfactory perfor- mance. Distant supervision (Mintz et al., 2009) instead uses training examples from a knowl- edge base, guaranteeing a large amount of (pop- ular) relation examples without human interven- tion, which can be used effectively by neural net- works (Lin et al., 2016; Glass et al., 2018). How- ever, even with this technique, approaching RE as a classification task presents several limitations: (1) distant supervision models are not accurate in extracting relations with a long-tailed distribution, because they typically have a small set of instances in knowledge bases; (2) in most domains, relation types are very specific and only a few examples of each relation are available; (3) these models can- …

Content truncated to 500 words