Improving Relation Extraction with Knowledge-attention

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 229–239, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 229 Improving Relation Extraction with Knowledge-attention Pengfei Li1, Kezhi Mao1∗, Xuefeng Yang3, and Qi Li2 1School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 2Interdisciplinary Graduate School, Nanyang Technological University, Singapore 3ZhuiYi Technology, Shenzhen, China {pli006,ekzmao,liqi0024}@ntu.edu.sg, ryan@wezhuiyi.com Abstract While attention mechanisms have been proven to be effective in many NLP tasks, major- ity of them are data-driven. We propose a novel knowledge-attention encoder which incorporates prior knowledge from external lexical resources into deep neural networks for relation extraction task. Furthermore, we present three effective ways of integrat- ing knowledge-attention with self-attention to maximize the utilization of both knowledge and data. The proposed relation extraction sys- tem is end-to-end and fully attention-based. Experiment results show that the proposed knowledge-attention mechanism has comple- mentary strengths with self-attention, and our integrated models outperform existing CNN, RNN, and self-attention based models. State- of-the-art performance is achieved on TA- CRED, a complex and large-scale relation ex- traction dataset. 1 Introduction Relation extraction aims to detect the semantic relationship between two entities in a sentence. For example, given the sentence: “James Dobson has resigned as chairman of Focus On The Family, which he founded thirty years ago.”, the goal is to recognize the organization-founder relation held between “Focus On The Family” and “James Dobson”. The various rela- tions between entities extracted from large-scale unstructured texts can be used for ontology and knowledge base population (Chen et al., 2018a; Fossati et al., 2018), as well as facilitating down- stream tasks that requires relational understand- ing of texts such as question answering (Yu et al., 2017) and dialogue systems (Young et al., 2018). Traditional feature-based and kernel-based ap- proaches require extensive feature engineer- ing (Suchanek et al., 2006; Qian et al., 2008; Rink ∗Corresponding author. and Harabagiu, 2010). Deep neural networks such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have the abil- ity of exploring more complex semantics and ex- tracting features automatically from raw texts for relation extraction tasks (Xu et al., 2016; Vu et al., 2016; Lee et al., 2017). Recently, attention mech- anisms have been introduced to deep neural net- works to improve their performance (Zhou et al., 2016; Wang et al., 2016; Zhang et al., 2017). Es- pecially, the Transformer proposed by Vaswani et al. (2017) is based solely on self-attention and has demonstrated better performance than tradi- tional RNNs (Bilan and Roth, 2018; Verga et al., 2018). However, deep neural networks normally require sufficient labeled data to train their numer- ous model parameters. The scarcity or low qual- ity of training data will limit the model’s ability to recognize complex relations and also cause over- fitting issue. A recent study (Li and Mao, 2019) shows that incorporating prior knowledge from external lex- ical resources into deep neural network can re- duce the reliance on …

Content truncated to 500 words