Relation Extraction with Word Graphs from N-grams

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2860–2868 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2860 Relation Extraction with Word Graphs from N-grams Han Qin♠∗, Yuanhe Tian♥∗, Yan Song♠† ♠The Chinese University of Hong Kong (Shenzhen) ♥University of Washington ♠hanqin@link.cuhk.edu.cn ♥yhtian@uw.edu ♠songyan@cuhk.edu.cn Abstract Most recent studies for relation extraction (RE) leverage the dependency tree of the in- put sentence to incorporate syntax-driven con- textual information to improve model perfor- mance, with little attention paid to the limi- tation where high-quality dependency parsers in most cases unavailable, especially for in- domain scenarios. To address this limitation, in this paper, we propose attentive graph con- volutional networks (A-GCN) to improve neu- ral RE methods with an unsupervised manner to build the context graph, without relying on the existence of a dependency parser. Specif- ically, we construct the graph from n-grams extracted from a lexicon built from pointwise mutual information (PMI) and apply attention over the graph. Therefore, different word pairs from the contexts within and across n-grams are weighted in the model and facilitate RE ac- cordingly. Experimental results with further analyses on two English benchmark datasets for RE demonstrate the effectiveness of our ap- proach, where state-of-the-art performance is observed on both datasets.1 1 Introduction Recently, neural models (Zeng et al., 2014; Zhang and Wang, 2015; Xu et al., 2015; dos Santos et al., 2015; Zhang et al., 2015; Wang et al., 2016; Zhou et al., 2016; Zhang et al., 2017; Soares et al., 2019) with powerful encoders (e.g., Transformers) have achieved promising performance for relation ex- traction (RE), for the reason that the encoders are superior in capturing contextual information and thus allow RE systems to better understand the text and correctly identify the relations between entities in the given text. To further improve the ability of RE models to understand the context, many stud- ies (Xu et al., 2016; Zhang et al., 2018; Guo et al., *Equal contribution. †Corresponding author. 1The code involved in this paper are released at https: //github.com/cuhksz-nlp/RE-NGCN. 2019; Sun et al., 2020; Yu et al., 2020; Mandya et al., 2020; Tian et al., 2021d; Chen et al., 2021) leverage extra resources, such as auto-parsed word dependency, through graph-based approaches, e.g., graph convolutional networks (GCN). In doing so, such studies learn the long-distance connections among useful words from the dependency tree and extract relations between entity pairs accordingly. However, in doing so, dependency parsers required by their approaches are not always available. In this dilemma, one needs another way to extract useful word connections to build the graph for GCN-based models, whereas limited attentions from previous studies have been paid to this alternative. In this paper, we propose attentive GCN (A- GCN) for relation extraction, where its input graph is built based on n-grams extracted with unsuper- vised methods i.e., pointwise mutual information (PMI), rather than an existing dependency parser. Specifically, two types of edges in the graph are introduced to model word connections within and across n-grams and …

Content truncated to 500 words