Two Training Strategies for Improving Relation Extraction over Universal Graph

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 3673–3684 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 3673 Two Training Strategies for Improving Relation Extraction over Universal Graph Qin Dai1, Naoya Inoue2, Ryo Takahashi1,3, Kentaro Inui1,3 1Tohoku University, Japan 2Stony Brook University, USA 3RIKEN Center for Advanced Intelligence Project, Japan {daiqin, naoya-i, ryo.t, inui}@ecei.tohoku.ac.jp Abstract This paper explores how the Distantly Super- vised Relation Extraction (DS-RE) can bene- fit from the use of a Universal Graph (UG), the combination of a Knowledge Graph (KG) and a large-scale text collection. A straight- forward extension of a current state-of-the-art neural model for DS-RE with a UG may lead to degradation in performance. We first report that this degradation is associated with the dif- ficulty in learning a UG and then propose two training strategies: (1) Path Type Adaptive Pre- training, which sequentially trains the model with different types of UG paths so as to pre- vent the reliance on a single type of UG path; and (2) Complexity Ranking Guided Attention mechanism, which restricts the attention span according to the complexity of a UG path so as to force the model to extract features not only from simple UG paths but also from complex ones. Experimental results on both biomedi- cal and NYT10 datasets prove the robustness of our methods and achieve a new state-of- the-art result on the NYT10 dataset. The code and datasets used in this paper are avail- able at https://github.com/baodaiqin/ UGDSRE. 1 Introduction Relation Extraction (RE) is an important task in Natural Language Processing (NLP). RE aims to turn unstructured texts into structured Knowledge Graph (KG), which is typically stored as (e1, r, e2) triplets, where e1 is a head entity, r is a relation and e2 is a tail entity, such as (aspirin, may treat, pain) and (Guy Maddin, place lived, Winnpeg). RE can be formulated as a classification task to pre- dict a predefined relation r from entity pair (e1, e2) annotated evidences. One obstacle that is encountered when building a RE system is the generation of a large amount of manually annotated training instances, which is ex- pensive and time-consuming. For coping with this difficulty, Mintz et al. (2009) propose Distant Su- pervision (DS) to automatically generate training samples via linking KGs to texts. They assume that if (e1, r, e2) is in a KG, then all sentences that con- tain (e1, e2) (hereafter, sentence evidences) express the relation r. It is well known that the DS as- sumption is too strong and inevitably accompanies the wrong labeling problem, such as the sentence evidences (1b and 2) below, which fail to express may treat and place lived relation respectively. (1) a. Aspirine1 is widely used for short-term treatment of paine2, fever or colds. b. The tumor was remarkably large in size, and paine2 unrelieved by aspirine1. (2) He is now finishing a documentary about Winnipege2, the final installment of a per- sonal trilogy that began …

Content truncated to 500 words