End-to-End Neural Relation Extraction with Global Optimization

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1730–1740 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics End-to-End Neural Relation Extraction with Global Optimization Meishan Zhang1 and Yue Zhang2 and Guohong Fu1 1. School of Computer Science and Technology, Heilongjiang University, China 2. Singapore University of Technology and Design mason.zms@gmail.com, yue zhang@sutd.edu.sg, ghfu@hotmail.com Abstract Neural networks have shown promising results for relation extraction. State-of- the-art models cast the task as an end-to- end problem, solved incrementally using a local classifier. Yet previous work us- ing statistical models have demonstrated that global optimization can achieve better performances compared to local classifica- tion. We build a globally optimized neural model for end-to-end relation extraction, proposing novel LSTM features in order to better learn context representations. In addition, we present a novel method to in- tegrate syntactic information to facilitate global learning, yet requiring little back- ground on syntactic grammars thus being easy to extend. Experimental results show that our proposed model is highly effec- tive, achieving the best performances on two standard benchmarks. 1 Introduction Extracting entities (Florian et al., 2006, 2010) and relations (Zhao and Grishman, 2005; Jiang and Zhai, 2007; Sun et al., 2011; Plank and Mos- chitti, 2013) from unstructured texts have been two central tasks in information extraction (Grish- man, 1997; Doddington et al., 2004). Traditional approaches to relation extraction take entity recog- nition as a predecessor step in a pipeline (Zelenko et al., 2003; Chan and Roth, 2011), predicting re- lations between given entities. In recent years, there has been a surge of inter- est in performing end-to-end relation extraction, jointly recognizing entities and relations given free text inputs (Li and Ji, 2014; Miwa and Sasaki, 2014; Miwa and Bansal, 2016; Gupta et al., 2016). End-to-end learning prevents error propagation in the pipeline approach, and allows cross-task de- pendencies to be modeled explicitly for entity recognition. As a result, it gives better relation ex- traction accuracies compared to pipelines. Miwa and Bansal (2016) were among the first to use neural networks for end-to-end relation extrac- tion, showing highly promising results. In partic- ular, they used bidirectional LSTM (Graves et al., 2013) to learn hidden word representations under a sentential context, and further leveraged tree- structured LSTM (Tai et al., 2015) to encode syn- tactic information, given the output of a parser. The resulting representations are then used for making local decisions for entity and relation ex- traction incrementally, leading to much improved results compared with the best statistical model (Li and Ji, 2014). This demonstrates the strength of neural representation learning for end-to-end rela- tion extraction. On the other hand, Miwa and Bansal (2016)’s model is trained locally, without considering struc- tural correspondences between incremental deci- sions. This is unlike existing statistical methods, which utilize well-studied structured prediction methods to address the problem (Li and Ji, 2014; Miwa and Sasaki, 2014). As has been commonly understood, learning local decisions for structured prediction can lead to label bias (Lafferty et al., …

Content truncated to 500 words