FSUIE: A Novel Fuzzy Span Mechanism for Universal Information Extraction

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 16318–16333 July 9-14, 2023 ©2023 Association for Computational Linguistics FSUIE: A Novel Fuzzy Span Mechanism for Universal Information Extraction Tianshuo Peng1,†, Zuchao Li1,2,†,∗, Lefei Zhang1,2, Bo Du1,2 and Hai Zhao3 1National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, Wuhan, 430072, P. R. China 2Hubei Luojia Laboratory, Wuhan 430072, P. R. China 3Department of Computer Science and Engineering, Shanghai Jiao Tong University {pengts,zcli-charlie,zhanglefei,dubo}@whu.edu.cn, zhaohai@cs.sjtu.edu.cn Abstract Universal Information Extraction (UIE) has been introduced as a unified framework for var- ious Information Extraction (IE) tasks and has achieved widespread success. Despite this, UIE models have limitations. For example, they rely heavily on span boundaries in the data during training, which does not reflect the reality of span annotation challenges. Slight adjustments to positions can also meet requirements. Addi- tionally, UIE models lack attention to the lim- ited span length feature in IE. To address these deficiencies, we propose the Fuzzy Span Uni- versal Information Extraction (FSUIE) frame- work. Specifically, our contribution consists of two concepts: fuzzy span loss and fuzzy span attention. Our experimental results on a series of main IE tasks show significant improvement compared to the baseline, especially in terms of fast convergence and strong performance with small amounts of data and training epochs. These results demonstrate the effectiveness and generalization of FSUIE in different tasks, set- tings, and scenarios. 1 Introduction Information Extraction (IE) is focused on extract- ing predefined types of information from unstruc- tured text sources, such as Named Entity Recog- nition (NER), Relationship Extraction (RE), and Sentiment Extraction (SE). To uniformly model the various IE tasks under a unified framework, a generative Universal Information Extraction (UIE) was proposed in (Lu et al., 2022) and has achieved widespread success on various IE datasets and benchmarks. Due to the necessity of a powerful ∗Corresponding author. † Equal contribution. This work was supported by the Fundamental Research Funds for the Central Universities (No. 2042023kf0133), the Special Fund of Hubei Luojia Laboratory under Grant 220100014 and the National Science Fund for Distinguished Young Scholars un- der Grant 62225113. Hai Zhao was funded by the Key Projects of National Natural Science Foundation of China (U1836222 and 61733011). Annotator #1: VEHICLE On the 3rd September evening, I saw a yellow sports car ... Annotator #2: VEHICLE On the 3rd September evening, I saw a yellow sports car ... Annotator #3: VEHICLE On the 3rd September evening, I saw a yellow sports car ... Figure 1: An example of annotation ambiguity for “ve- hicle" entity in sentence “On the 3rd September evening, I saw a yellow sports car drive past my house". generative pre-training model for the generative UIE, the time overhead is extensive and the effi- ciency is not satisfactory. For this reason, this paper examines span-based UIE to unify various IE tasks, conceptualizing IE tasks as predictions of spans. However, UIE models still have some limitations. First, as it is …

Content truncated to 500 words