Bootstrapping Relation Extractors using Syntactic Search by Examples

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1491–1503 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1491 Bootstrapping Relation Extractors using Syntactic Search by Examples Matan Eyal1 Asaf Amrami1, 2 Hillel Taub-Tabib1 Yoav Goldberg1, 2 1Allen Institute for AI, Tel Aviv, Israel 2Bar Ilan University, Ramat-Gan, Israel matane,asafa,hillelt,yoavg@allenai.org Abstract The advent of neural-networks in NLP brought with it substantial improvements in supervised relation extraction. However, obtaining a suffi- cient quantity of training data remains a key challenge. In this work we propose a pro- cess for bootstrapping training datasets which can be performed quickly by non-NLP-experts. We take advantage of search engines over syntactic-graphs (Such as Shlain et al. (2020)) which expose a friendly by-example syntax. We use these to obtain positive examples by searching for sentences that are syntactically similar to user input examples. We apply this technique to relations from TACRED and Do- cRED and show that the resulting models are competitive with models trained on manually annotated data and on data obtained from dis- tant supervision. The models also outperform models trained using NLG data augmentation techniques. Extending the search-based ap- proach with the NLG method further improves the results. 1 Introduction The goal of Relation Extraction (RE) is to find and classify instances of certain relations in raw text. We denote a binary relation instance, i.e. a relation instance with two arguments, with a tu- ple x = (s, e1, e2, r), where s = [w0 · · · wn] is a sequence of sentence tokens, e1, e2 are entity men- tions within s corresponding to the first and second relation argument, respectively, and r ∈ R ∪ {∅} is a relation label from a set of predefined relations of interest, or an indication of ‘no-relation’. In bi- nary classification our goal is to classify whether, according to s, the entity mentions, e1 and e2, sat- isfy r, the relation label. For such classification we require a training dataset X, comprised of Xp, a set of positive examples, representing the relation of interest, and Xn, a set of negatives examples. The success of recent papers (Soares et al., 2019; Murty et al., 2020) in supervised RE is fueled by ad- vances in deep learning, but also, crucially, by the availability of a large training set such as TACRED (Zhang et al., 2017), containing tens of thousands of training examples. For most relations of interest, such training data is not available. In this work we examine methods to inexpen- sively construct Xp and Xn, in cases where a training set is not available. We are especially interested in constructing the positive set, Xp. In contrast to common NLP tasks like POS tag- ging, entity extraction and dependency parsing, the task of relation extraction exhibits a much larger degree of label sparsity. For some relations, even when considering only sentences with entities of the relevant types, the ratio between positive and negative examples is highly skewed toward …

Content truncated to 500 words