Exploiting the Syntax-Model Consistency for Neural Relation Extraction

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8021–8032 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 8021 Exploiting the Syntax-Model Consistency for Neural Relation Extraction Amir Pouran Ben Veyseh1, Franck Dernoncourt2, Dejing Dou1 and Thien Huu Nguyen1,3 1Department of Computer and Information Science, University of Oregon, Eugene, Oregon, USA 2Adobe Research, San Jose, CA, USA 3VinAI Research, Hanoi, Vietnam {apouranb, dou, thien}@cs.uoregon.edu franck.dernoncourt@adobe.com Abstract This paper studies the task of Relation Extrac- tion (RE) that aims to identify the semantic re- lations between two entity mentions in text. In the deep learning models for RE, it has been beneficial to incorporate the syntactic struc- tures from the dependency trees of the input sentences. In such models, the dependency trees are often used to directly structure the network architectures or to obtain the depen- dency relations between the word pairs to in- ject the syntactic information into the models via multi-task learning. The major problems with these approaches are the lack of general- ization beyond the syntactic structures in the training data or the failure to capture the syn- tactic importance of the words for RE. In or- der to overcome these issues, we propose a novel deep learning model for RE that uses the dependency trees to extract the syntax-based importance scores for the words, serving as a tree representation to introduce syntactic in- formation into the models with greater gener- alization. In particular, we leverage Ordered- Neuron Long-Short Term Memory Networks (ON-LSTM) to infer the model-based impor- tance scores for RE for every word in the sen- tences that are then regulated to be consistent with the syntax-based scores to enable syn- tactic information injection. We perform ex- tensive experiments to demonstrate the effec- tiveness of the proposed method, leading to the state-of-the-art performance on three RE benchmark datasets. 1 Introduction One of the fundamental tasks in Information Extrac- tion (IE) is Relation Extraction (RE) where the goal is to find the semantic relationships between two entity mentions in text. Due to its importance, RE has been studied extensively in the literature. The recent studies on RE has focused on deep learning to develop methods to automatically induce sen- tence representations from data (Zeng et al., 2014; Nguyen and Grishman, 2015a; Verga et al., 2018). A notable insight in these recent studies is that the syntactic trees of the input sentences (i.e., the de- pendency trees) can provide effective information for the deep learning models, leading to the state- of-the-art performance for RE recently (Xu et al., 2015; Guo et al., 2019; Tran et al., 2019). In par- ticular, the previous deep learning models for RE has mostly exploited the syntactic trees to structure the network architectures according to the word connections presented in the trees (e.g., perform- ing Graph Convolutional Neural Networks (GCN) over the dependency trees (Zhang et al., 2018)). Unfortunately, these models might not be able to generalize well as the tree structures of the training data might …

Content truncated to 500 words