UniEX: An Effective and Efficient Framework for Unified Information Extraction via a Span-extractive Perspective

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 16424–16440 July 9-14, 2023 ©2023 Association for Computational Linguistics UniEX: An Effective and Efficient Framework for Unified Information Extraction via a Span-extractive Perspective Ping Yang1∗ Junyu Lu12∗ Ruyi Gan1∗ Junjie Wang3 Yuxiang Zhang3 Jiaxing Zhang1† Pingjian Zhang2† 1International Digital Economy Academy 2South China University of Technology 3Waseda University {yangping, lujunyu, ganruyi}@idea.edu.cn wjj1020181822@toki.waseda.jp, joel0495@asagi.waseda.jp pjzhang@scut.edu.cn Abstract We propose a new paradigm for universal in- formation extraction (IE) that is compatible with any schema format and applicable to a list of IE tasks, such as named entity recogni- tion, relation extraction, event extraction and sentiment analysis. Our approach converts the text-based IE tasks as the token-pair problem, which uniformly disassembles all extraction tar- gets into joint span detection, classification and association problems with a unified extractive framework, namely UniEX. UniEX can syn- chronously encode schema-based prompt and textual information, and collaboratively learn the generalized knowledge from pre-defined information using the auto-encoder language models. We develop a traffine attention mech- anism to integrate heterogeneous factors in- cluding tasks, labels and inside tokens, and obtain the extraction target via a scoring ma- trix. Experiment results show that UniEX can outperform generative universal IE models in terms of performance and inference-speed on 14 benchmarks IE datasets with the supervised setting. The state-of-the-art performance in low-resource scenarios also verifies the trans- ferability and effectiveness of UniEX. 1 Introduction Information extraction (IE) aims at automatically extracting structured information from unstruc- tured textual sources, covering a wide range of subtasks such as named entity recognition, relation extraction, semantic role labeling, and sentiment analysis (Muslea et al., 1999; Grishman, 2019). However, the variety of subtasks build the isola- tion zones between each other and form their own dedicated models. Fig 1 (a) presents that the popu- lar IE approaches handle structured extraction by the addition of task-specific layers on top of pre- trained language models (LMs) and a subsequent *Equal Contribution. †Corresponding Author. Encoder Decoder Text T5, BART, ... Encoder + CRF Encoder + SpanExtractor …... Decompose (a) Task-specialized IE (b) Generative Universal IE (TANL, UIE) Encoder + Triaffine BERT, ALBERT, ... Decode (c) Extractive Universal IE (UniEX) Task A Extraction Target A Task A, B, C, ... Text Extraction Target A, B, C, ... Unified Input Format Structured Language Schema-based prompt Task A, B, C, ... Text Unified Input Format Schema-based prompt Extraction Target A, B, C, ... Scoring Matrix Figure 1: (a) Task-specific IE methods: isolated struc- tures and schemas. (b) Typical generative universal IE: unified modeling via text or structure generation. (c) Our extractive universal IE: unified modeling via traf- fine attention mechanism and auto-encoder LMs. fine-tuning of the conjoined model (Lample et al., 2016; Luo et al., 2020; Wei et al., 2020; Ye et al., 2022). The isolated architectures and chaotic situ- ation prevents enhancements from one task from being applied to another, which hinders the effec- tive latent semantics sharing such as label names, and suffer from …

Content truncated to 500 words