Entity, Relation, and Event Extraction with Contextualized Span Representations

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5784–5789, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5784 Entity, Relation, and Event Extraction with Contextualized Span Representations David Wadden† Ulme Wennberg† Yi Luan‡ Hannaneh Hajishirzi†∗ †Paul G. Allen School of Computer Science & Engineering, University of Washington ‡Google AI Language ∗Allen Institute for Artificial Intelligence {dwadden,ulme,hannaneh}@cs.washington.edu luanyi@google.com Abstract We examine the capabilities of a unified, multi- task framework for three information extrac- tion tasks: named entity recognition, rela- tion extraction, and event extraction. Our framework (called DYGIE++) accomplishes all tasks by enumerating, refining, and scoring text spans designed to capture local (within- sentence) and global (cross-sentence) con- text. Our framework achieves state-of-the- art results across all tasks, on four datasets from a variety of domains. We perform ex- periments comparing different techniques to construct span representations. Contextual- ized embeddings like BERT perform well at capturing relationships among entities in the same or adjacent sentences, while dynamic span graph updates model long-range cross- sentence relationships. For instance, propagat- ing span representations via predicted coref- erence links can enable the model to disam- biguate challenging entity mentions. Our code is publicly available at https://github. com/dwadden/dygiepp and can be easily adapted for new tasks or datasets. 1 Introduction Many information extraction tasks – including named entity recognition, relation extraction, event extraction, and coreference resolution – can benefit from incorporating global context across sentences or from non-local dependencies among phrases. For example, knowledge of a coreference relation- ship can provide information to help infer the type of a difficult-to-classify entity mention. In event extraction, knowledge of the entities present in a sentence can provide information that is useful for predicting event triggers. To model global context, previous works have used pipelines to extract syntactic, discourse, and other hand-engineered features as inputs to struc- tured prediction models (Li et al., 2013; Yang and BERT Relation propagation Coreference propagation Graph propagation Events Entities Relations Span enumeration Event propagation Coreference Auxiliary Sentence Sentence … … Sentence Figure 1: Overview of our framework: DYGIE++. Shared span representations are constructed by refin- ing contextualized word embeddings via span graph updates, then passed to scoring functions for three IE tasks. Mitchell, 2016; Li and Ji, 2014) and neural scor- ing functions (Nguyen and Nguyen, 2019), or as a guide for the construction of neural architectures (Peng et al., 2017; Zhang et al., 2018; Sha et al., 2018; Christopoulou et al., 2018). Recent end-to- end systems have achieved strong performance by dynmically constructing graphs of spans whose edges correspond to task-specific relations (Luan et al., 2019; Lee et al., 2018; Qian et al., 2018). Meanwhile, contextual language models (Dai and Le, 2015; Peters et al., 2017, 2018; Devlin et al., 2018) have proven successful on a range of natural language processing tasks (Bowman et al., 2015; Sang and De Meulder, 2003; Rajpurkar et al., 2016). Some of these models are also capable of …

Content truncated to 500 words