Modeling Instance Interactions for Joint Information Extraction with Neural High-Order Conditional Random Field

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 13695–13710 July 9-14, 2023 ©2023 Association for Computational Linguistics Modeling Instance Interactions for Joint Information Extraction with Neural High-Order Conditional Random Field Zixia Jia1,2∗, Zhaohui Yan2, Wenjuan Han3, Zilong Zheng1†, Kewei Tu2† 1 Beijing Institute for General Artificial Intelligence (BIGAI), Beijing, China 2 ShanghaiTech University, Shanghai, China 3 Beijing Jiaotong University, Beijing, China {jiazixia,zlzheng}@bigai.ai {yanzhh,tukw}@shanghaitech.edu.cn, wjhan@bjtu.edu.cn Abstract Prior works on joint Information Extraction (IE) typically model instance (e.g., event trig- gers, entities, roles, relations) interactions by representation enhancement, type dependen- cies scoring, or global decoding. We find that the previous models generally consider binary type dependency scoring of a pair of instances, and leverage local search such as beam search to approximate global solutions. To better inte- grate cross-instance interactions, in this work, we introduce a joint IE framework (CRFIE) that formulates joint IE as a high-order Condi- tional Random Field. Specifically, we design binary factors and ternary factors to directly model interactions between not only a pair of instances but also triplets. Then, these factors are utilized to jointly predict labels of all in- stances. To address the intractability problem of exact high-order inference, we incorporate a high-order neural decoder that is unfolded from a mean-field variational inference method, which achieves consistent learning and infer- ence. The experimental results show that our approach achieves consistent improvements on three IE tasks compared with our baseline and prior work. 1 Introduction Information extraction (IE) has long been consid- ered a fundamental challenge for various down- stream natural language understanding tasks, such as knowledge graph construction and reading com- prehension, etc. The goal is to identify and extract structured information from unstructured natural language text, such that both users and machines can easily comprehend the entities, relations, and events within the text. Typically, IE consists of a series of different tasks to recognize entities, connect coreferences, ∗This work was conducted when Zixia Jia was a research intern at BIGAI. †Correspondence to Zilong Zheng and Kewei Tu. PER PER PER PER-SOC PER-SOC Victim Victim Life:Die The boys ’ mother and two other brothers were killed . I 'm Kelly Wallace , CNN , reporting from Netanyahu , Israel . PER ORG GPE GPE ORG-AFF PHYS PART-WHOLE Figure 1: Example annotations of entity recognition (e.g., Blue Red Orange Victim PER-SOC PER), relation extraction (e.g., Blue Red Orange Victim PER-SOC PER ) and event extraction tasks (e.g., Blue Red Orange Victim PER-SOC PER Life:Die PART-WHOLE GPE and Blue Red Orange Victim PER-SOC PER ). extract relations, detect events, and so on. Con- ventional IE schemes commonly treat different IE tasks separately, while neglecting cross-instance (e.g., event triggers, entities, roles, relations) or cross-task dependencies. Such isolated learning and inference schemes lead to severely insufficient knowledge capturing and inefficient model con- structions. Intuitively, predictions of different IE instances from the same or different tasks can influ- ence each other. For example, a relation between two entities would restrict the types …

Content truncated to 500 words