Injecting Knowledge Base Information into End-to-End Joint Entity and Relation Extraction and Coreference Resolution

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1952–1957 August 1–6, 2021. ©2021 Association for Computational Linguistics 1952 Injecting Knowledge Base Information into End-to-End Joint Entity and Relation Extraction and Coreference Resolution Severine Verlinden∗, Klim Zaporojets∗, Johannes Deleu, Thomas Demeester, Chris Develder Ghent University – imec, IDLab Ghent, Belgium {first name.last name}@ugent.be Abstract We consider a joint information extraction (IE) model, solving named entity recognition, coreference resolution and relation extraction jointly over the whole document. In particu- lar, we study how to inject information from a knowledge base (KB) in such IE model, based on unsupervised entity linking. The used KB entity representations are learned from either (i) hyperlinked text documents (Wikipedia), or (ii) a knowledge graph (Wikidata), and ap- pear complementary in raising IE performance. Representations of corresponding entity link- ing (EL) candidates are added to text span rep- resentations of the input document, and we ex- periment with (i) taking a weighted average of the EL candidate representations based on their prior (in Wikipedia), and (ii) using an attention scheme over the EL candidate list. Results demonstrate an increase of up to 5% F1-score for the evaluated IE tasks on two datasets. Despite a strong performance of the prior-based model, our quantitative and quali- tative analysis reveals the advantage of using the attention-based approach. 1 Introduction Information extraction (IE) comprises several sub- tasks, e.g., named entity recognition (NER), coref- erence resolution (coref), relation extraction (RE). State-of-the-art results mainly report performance on single tasks, usually solving them on a sentence level (especially NER, RE). However, in practice, IE system decisions should be consistent on the document level, e.g., when processing news arti- cles to automatically link entities (aside from po- tentially learning, e.g., new relations). Yet, the challenge of solving the tasks jointly on a docu- ment level has not received as much attention and remains hard (Durrett and Klein, 2014; Yao et al., 2019; Zaporojets et al., 2021). ∗Equal contribution On the other hand, it is well established that IE models benefit from incorporating background information of knowledge bases (KBs). Still, so far this has been shown from the perspective of solving individual tasks such as relation classification or entity typing (e.g., Peters et al. (2019); Liu et al. (2020)). Integrating KBs in joint models, realizing and analyzing the more complex end-to-end setting, has been left unexplored. In terms of the nature of KBs adopted in IE, cur- rent approaches use either (i) structured knowledge graphs comprising (subj,rel,obj) triples, e.g., Wikidata (Yang and Mitchell, 2017; Han et al., 2018; Zhang et al., 2019), or (ii) textual descriptions, usually in hyperlinked documents, e.g., Wikipedia (Martins et al., 2019; Yamada et al., 2020). It has not been established to what extent KB-text and KB-graph entity representations com- plement each other in boosting IE performance. We address both research gaps of (a) integrating KB information into a joint end-to-end IE model for solving named entity recognition, coreference resolution and relation extraction, and (b) analyz- ing what KB representation is …

Content truncated to 500 words