Effective Attention Modeling for Neural Relation Extraction

Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 603–612 Hong Kong, China, November 3-4, 2019. c⃝2019 Association for Computational Linguistics 603 Effective Attention Modeling for Neural Relation Extraction Tapas Nayak and Hwee Tou Ng Department of Computer Science National University of Singapore nayakt@u.nus.edu, nght@comp.nus.edu.sg Abstract Relation extraction is the task of determin- ing the relation between two entities in a sen- tence. Distantly-supervised models are popu- lar for this task. However, sentences can be long and two entities can be located far from each other in a sentence. The pieces of ev- idence supporting the presence of a relation between two entities may not be very direct, since the entities may be connected via some indirect links such as a third entity or via co- reference. Relation extraction in such scenar- ios becomes more challenging as we need to capture the long-distance interactions among the entities and other words in the sentence. Also, the words in a sentence do not contribute equally in identifying the relation between the two entities. To address this issue, we propose a novel and effective attention model which incorporates syntactic information of the sen- tence and a multi-factor attention mechanism. Experiments on the New York Times corpus show that our proposed model outperforms prior state-of-the-art models. 1 Introduction Relation extraction from unstructured text is an important task to build knowledge bases (KB) au- tomatically. Banko et al. (2007) used open in- formation extraction (Open IE) to extract relation triples from sentences where verbs were consid- ered as the relation, whereas supervised informa- tion extraction systems extract a set of pre-defined relations from text. Mintz et al. (2009), Riedel et al. (2010), and Hoffmann et al. (2011) proposed distant supervision to generate the training data for sentence-level relation extraction, where rela- tion tuples (two entities and the relation between them) from a knowledge base such as Freebase (Bollacker et al., 2008) were mapped to free text (Wikipedia articles or New York Times articles). The idea is that if a sentence contains both enti- ties of a tuple, it is chosen as a training sentence of that tuple. Although this process can generate some noisy training instances, it can give a signif- icant amount of training data which can be used to build supervised models for this task. Mintz et al. (2009), Riedel et al. (2010), and Hoffmann et al. (2011) proposed feature-based learning models and used entity tokens and their nearby tokens, their part-of-speech tags, and other linguistic features to train their models. Recently, many neural network-based models have been proposed to avoid feature engineering. Zeng et al. (2014, 2015) used convolutional neural networks (CNN) with max-pooling to find the relation be- tween two given entities. Though these models have been shown to perform reasonably well on distantly supervised data, they sometimes fail to find the relation when sentences are long and enti- ties are located far from each other. CNN mod- els with max-pooling have limitations in under- standing the semantic similarity …

Content truncated to 500 words