UTC-IE: A Unified Token-pair Classification Architecture for Information Extraction

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4096–4122 July 9-14, 2023 ©2023 Association for Computational Linguistics UTC-IE: A Unified Token-pair Classification Architecture for Information Extraction Hang Yan∗, Yu Sun∗, Xiaonan Li, Yunhua Zhou, Xuanjing Huang, Xipeng Qiu† Shanghai Key Laboratory of Intelligent Information Processing, Fudan University School of Computer Science, Fudan University {hyan19,lixn20,zhouyh20,xjhuang,xpqiu}@fudan.edu.cn yusun21@m.fudan.edu.cn Abstract Information Extraction (IE) spans several tasks with different output structures, such as named entity recognition, relation extraction and event extraction. Previously, those tasks were solved with different models because of diverse task output structures. Through re-examining IE tasks, we find that all of them can be inter- preted as extracting spans and span relations. They can further be decomposed into token- pair classification tasks by using the start and end token of a span to pinpoint the span, and using the start-to-start and end-to-end token pairs of two spans to determine the relation. Based on the reformulation, we propose a Unified Token-pair Classification architecture for Information Extraction (UTC-IE), where we introduce Plusformer on top of the token- pair feature matrix. Specifically, it models axis-aware interaction with plus-shaped self- attention and local interaction with Convolu- tional Neural Network over token pairs. Ex- periments show that our approach outperforms task-specific and unified models on all tasks in 10 datasets, and achieves better or comparable results on 2 joint IE datasets. Moreover, UTC- IE speeds up over state-of-the-art models on IE tasks significantly in most datasets, which verifies the effectiveness of our architecture.1 1 Introduction Information Extraction (IE) aims to identify and classify structured information from unstructured texts (Andersen et al., 1992; Grishman, 2019). IE consists of a wide range of tasks, such as named entity recognition (NER), joint entity relation ex- traction (RE)2 and event extraction (EE) . In the last decade, many paradigms have been proposed to solve IE tasks, such as sequence label- ∗Equal contribution. †Corresponding author. 1Code is available at https://github.com/yhcc/utcie. 2Joint entity relation extraction aims to extract both entities and relations. In our paper, we call it relation extraction (RE) for simplicity. ing (McCallum and Li, 2003; Huang et al., 2015; Zheng et al., 2017; Yu et al., 2020a), span-based classification (Jiang et al., 2020; Yu et al., 2020b; Wang et al., 2021; Ye et al., 2022), MRC-based methods (Levy et al., 2017; Li et al., 2020; Liu et al., 2020) and generation-based methods (Zeng et al., 2018; Yan et al., 2021a; Hsu et al., 2022). The above work mainly concentrates on solving individ- ual tasks, but it is desired to unify all IE tasks with- out designing dedicated modules, as tackling all IE tasks with one model can facilitate knowledge sharing between different tasks. Therefore, various attempts have been made to unify all IE tasks with one model structure. Wadden et al. (2019); Lin et al. (2020); Nguyen et al. (2021) encode all IE tasks’ target structure as graphs and design graph-based methods to predict them; Paolini et al. (2021); Lu et …

Content truncated to 500 words