End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1105–1116, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures Makoto Miwa Toyota Technological Institute Nagoya, 468-8511, Japan makoto-miwa@toyota-ti.ac.jp Mohit Bansal Toyota Technological Institute at Chicago Chicago, IL, 60637, USA mbansal@ttic.edu Abstract We present a novel end-to-end neural model to extract entities and relations be- tween them. Our recurrent neural net- work based model captures both word se- quence and dependency tree substructure information by stacking bidirectional tree- structured LSTM-RNNs on bidirectional sequential LSTM-RNNs. This allows our model to jointly represent both entities and relations with shared parameters in a sin- gle model. We further encourage detec- tion of entities during training and use of entity information in relation extraction via entity pretraining and scheduled sam- pling. Our model improves over the state- of-the-art feature-based model on end-to- end relation extraction, achieving 12.1% and 5.7% relative error reductions in F1- score on ACE2005 and ACE2004, respec- tively. We also show that our LSTM- RNN based model compares favorably to the state-of-the-art CNN based model (in F1-score) on nominal relation classifica- tion (SemEval-2010 Task 8). Finally, we present an extensive ablation analysis of several model components. 1 Introduction Extracting semantic relations between entities in text is an important and well-studied task in in- formation extraction and natural language pro- cessing (NLP). Traditional systems treat this task as a pipeline of two separated tasks, i.e., named entity recognition (NER) (Nadeau and Sekine, 2007; Ratinov and Roth, 2009) and relation extraction (Zelenko et al., 2003; Zhou et al., 2005), but recent studies show that end-to-end (joint) modeling of entity and relation is impor- tant for high performance (Li and Ji, 2014; Miwa and Sasaki, 2014) since relations interact closely with entity information. For instance, to learn that Toefting and Bolton have an Organization- Affiliation (ORG-AFF) relation in the sentence Toefting transferred to Bolton, the entity informa- tion that Toefting and Bolton are Person and Orga- nization entities is important. Extraction of these entities is in turn encouraged by the presence of the context words transferred to, which indicate an employment relation. Previous joint models have employed feature-based structured learning. An alternative approach to this end-to-end relation ex- traction task is to employ automatic feature learn- ing via neural network (NN) based models. There are two ways to represent relations be- tween entities using neural networks: recur- rent/recursive neural networks (RNNs) and convo- lutional neural networks (CNNs). Among these, RNNs can directly represent essential linguis- tic structures, i.e., word sequences (Hammerton, 2001) and constituent/dependency trees (Tai et al., 2015). Despite this representation ability, for relation classification tasks, the previously re- ported performance using long short-term memory (LSTM) based RNNs (Xu et al., 2015b; Li et al., 2015) is worse than one using CNNs (dos Santos et al., 2015). These previous LSTM-based sys- tems mostly include limited linguistic structures and neural architectures, and do not model entities and relations jointly. We …

Content truncated to 500 words