Matching the Blanks: Distributional Similarity for Relation Learning

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2895–2905 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2895 Matching the Blanks: Distributional Similarity for Relation Learning Livio Baldini Soares Nicholas FitzGerald Jeffrey Ling∗ Tom Kwiatkowski Google Research {liviobs,nfitz,jeffreyling,tomkwiat}@google.com Abstract General purpose relation extractors, which can model arbitrary relations, are a core aspiration in information extraction. Efforts have been made to build general purpose extractors that represent relations with their surface forms, or which jointly embed surface forms with rela- tions from an existing knowledge graph. How- ever, both of these approaches are limited in their ability to generalize. In this paper, we build on extensions of Harris’ distributional hypothesis to relations, as well as recent ad- vances in learning text representations (specif- ically, BERT), to build task agnostic relation representations solely from entity-linked text. We show that these representations signifi- cantly outperform previous work on exemplar based relation extraction (FewRel) even with- out using any of that task’s training data. We also show that models initialized with our task agnostic representations, and then tuned on su- pervised relation extraction datasets, signifi- cantly outperform the previous methods on Se- mEval 2010 Task 8, KBP37, and TACRED. 1 Introduction Reading text to identify and extract relations be- tween entities has been a long standing goal in natural language processing (Cardie, 1997). Typ- ically efforts in relation extraction fall into one of three groups. In a first group, supervised (Kamb- hatla, 2004; GuoDong et al., 2005; Zeng et al., 2014), or distantly supervised relation extractors (Mintz et al., 2009) learn a mapping from text to relations in a limited schema. Forming a sec- ond group, open information extraction removes the limitations of a predefined schema by instead representing relations using their surface forms (Banko et al., 2007; Fader et al., 2011; Stanovsky et al., 2018), which increases scope but also leads ∗Work done as part of the Google AI residency. to an associated lack of generality since many sur- face forms can express the same relation. Finally, the universal schema (Riedel et al., 2013) em- braces both the diversity of text, and the concise nature of schematic relations, to build a joint rep- resentation that has been extended to arbitrary tex- tual input (Toutanova et al., 2015), and arbitrary entity pairs (Verga and McCallum, 2016). How- ever, like distantly supervised relation extractors, universal schema rely on large knowledge graphs (typically Freebase (Bollacker et al., 2008)) that can be aligned to text. Building on Lin and Pantel (2001)’s extension of Harris’ distributional hypothesis (Harris, 1954) to relations, as well as recent advances in learning word representations from observations of their contexts (Mikolov et al., 2013; Peters et al., 2018; Devlin et al., 2018), we propose a new method of learning relation representations directly from text. First, we study the ability of the Transformer neural network architecture (Vaswani et al., 2017) to encode relations between entity pairs, and we identify a method of representation that …

Content truncated to 500 words