A Joint Neural Model for Information Extraction with Global Features

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7999–8009 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 7999 A Joint Neural Model for Information Extraction with Global Features Ying Lin1, Heng Ji1, Fei Huang2, Lingfei Wu3 1University of Illinois at Urbana-Champaign 2Alibaba DAMO Academy 3IBM Research {yinglin8,hengji}@illinois.edu, f.huang@alibaba-inc.com, wuli@us.ibm.com Abstract Most existing joint neural models for Infor- mation Extraction (IE) use local task-specific classifiers to predict labels for individual in- stances (e.g., trigger, relation) regardless of their interactions. For example, a VICTIM of a DIE event is likely to be a VICTIM of an AT- TACK event in the same sentence. In order to capture such cross-subtask and cross-instance inter-dependencies, we propose a joint neural framework, ONEIE, that aims to extract the globally optimal IE result as a graph from an input sentence. ONEIE performs end-to-end IE in four stages: (1) Encoding a given sen- tence as contextualized word representations; (2) Identifying entity mentions and event trig- gers as nodes; (3) Computing label scores for all nodes and their pairwise links using local classifiers; (4) Searching for the globally op- timal graph with a beam decoder. At the de- coding stage, we incorporate global features to capture the cross-subtask and cross-instance interactions. Experiments show that adding global features improves the performance of our model and achieves new state-of-the-art on all subtasks. As ONEIE does not use any language-specific feature, we prove it can be easily applied to new languages or trained in a multilingual manner. Our code and models for English, Spanish and Chinese are publicly available for research purpose 1. 1 Introduction Information Extraction (IE) aims to extract struc- tured information from unstructured texts. It is a complex task comprised of a wide range of sub- tasks, such as named, nominal, and pronominal mention extraction, entity linking, entity corefer- ence resolution, relation extraction, event extrac- tion, and event coreference resolution. Early ef- forts typically perform IE in a pipelined fashion, 1 http://blender.cs.illinois.edu/software/ oneie which leads to the error propagation problem and disallows interactions among components in the pipeline. As a solution, some researchers propose joint inference and joint modeling methods to im- prove local prediction (Roth and Yih, 2004; Ji and Grishman, 2005; Ji et al., 2005; Sil and Yates, 2013; Li et al., 2014; Durrett and Klein, 2014; Miwa and Sasaki, 2014; Lu and Roth, 2015; Yang and Mitchell, 2016; Kirschnick et al., 2016). Due to the success of deep learning, neural models have been widely applied to various IE subtasks (Col- lobert et al., 2011; Chiu and Nichols, 2016; Chen et al., 2015; Lin et al., 2016). Recently, some ef- forts (Wadden et al., 2019; Luan et al., 2019) re- visit global inference approaches by designing neu- ral networks with embedding features to jointly model multiple subtasks. However, these methods use separate local task-specific classifiers in the final layer and do not explicitly model the inter- dependencies among tasks and instances. Figure 1 shows a real example where the local …

Content truncated to 500 words