Global Normalization of Convolutional Neural Networks for Joint Entity and Relation Classification

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1723–1729 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Global Normalization of Convolutional Neural Networks for Joint Entity and Relation Classification Heike Adel and Hinrich Sch¨utze Center for Information and Language Processing (CIS) LMU Munich, Germany heike@cis.lmu.de Abstract We introduce globally normalized convo- lutional neural networks for joint entity classification and relation extraction. In particular, we propose a way to utilize a linear-chain conditional random field out- put layer for predicting entity types and re- lations between entities at the same time. Our experiments show that global normal- ization outperforms a locally normalized softmax layer on a benchmark dataset. 1 Introduction Named entity classification (EC) and relation ex- traction (RE) are important topics in natural lan- guage processing. They are relevant, e.g., for pop- ulating knowledge bases or answering questions from text, such as “Where does X live?” Most approaches consider the two tasks inde- pendent from each other or treat them as a se- quential pipeline by first applying a named entity recognition tool and then classifying relations be- tween entity pairs. However, named entity types and relations are often mutually dependent. If the types of entities are known, the search space of possible relations between them can be reduced and vice versa. This can help, for example, to resolve ambiguities, such as in the case of “Mer- cedes”, which can be a person, organization and location. However, knowing that in the given context, it is the second argument for the rela- tion “live in” helps concluding that it is a loca- tion. Therefore, we propose a single neural net- work (NN) for both tasks. In contrast to joint train- ing and multitask learning, which calculate task- wise costs, we propose to learn a joint classifi- cation layer which is globally normalized on the outputs of both tasks. In particular, we train the NN parameters based on the loss of a linear-chain Anderson , 41 , was the chief Middle East correspondent for The Associated Press PER O O O O O O LOC O O ORG work_for live_in based_in Model inputs (query entity pairs) ⇒ Model outputs: (“Anderson”, “,”) ⇒ PER - N - O ...(“Anderson”, “41”) ⇒ PER - N - O (“Anderson”, “chief”) ⇒ PER - N - O ...(“Anderson”, “Middle East”) ⇒ PER - live in - LOC ...(“was”, “for”) ⇒ O - N - O (“for”, “The Associated Press”) ⇒ O - N - ORG Figure 1: Examples of our task conditional random field (CRF) (Lafferty et al., 2001). CRF layers for NNs have been introduced for token-labeling tasks like named entity recog- nition (NER) or part-of-speech tagging (Collobert et al., 2011; Lample et al., 2016; Andor et al., 2016). Instead of labeling each input token as in previous work, we model the joint entity and rela- tion classification problem as a sequence of length three for the CRF layer. In particular, we identify the types of two candidate …

Content truncated to 500 words