Packed Levitated Marker for Entity and Relation Extraction

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4904 - 4917 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Packed Levitated Marker for Entity and Relation Extraction Deming Ye1,2, Yankai Lin6, Peng Li6,7, Maosong Sun1,2,3,4,5∗ 1Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China 2Beijing National Research Center for Information Science and Technology 3International Innovation Center of Tsinghua University, Shanghai, China 4Jiangsu Collaborative Innovation Center for Language Ability, Xuzhou, China 5Institute Guo Qiang, Tsinghua University 6Pattern Recognition Center, WeChat AI 7Institute for AI Industry Research (AIR), Tsinghua University yedeming001@163.com Abstract Recent entity and relation extraction works focus on investigating how to obtain a bet- ter span representation from the pre-trained encoder. However, a major limitation of existing works is that they ignore the in- terrelation between spans (pairs). In this work, we propose a novel span representa- tion approach, named Packed Levitated Mark- ers (PL-Marker), to consider the interrela- tion between the spans (pairs) by strategically packing the markers in the encoder. In par- ticular, we propose a neighborhood-oriented packing strategy, which considers the neigh- bor spans integrally to better model the en- tity boundary information. Furthermore, for those more complicated span pair classifica- tion tasks, we design a subject-oriented pack- ing strategy, which packs each subject and all its objects to model the interrelation between the same-subject span pairs. The experimental results show that, with the enhanced marker feature, our model advances baselines on six NER benchmarks, and obtains a 4.1%-4.3% strict relation F1 improvement with higher speed over previous state-of-the-art models on ACE04 and ACE05. Our code and models are publicly available at https://github.com/ thunlp/PL-Marker. 1 Introduction Recently, pre-trained language models (PLMs) (De- vlin et al., 2019; Liu et al., 2019) have achieved significant improvements in Named Entity Recog- nition (NER, Luo et al. (2020); Fu et al. (2021)) and Relation Extraction (RE, Wadden et al. (2019); Zhou and Chen (2021)), two key sub-tasks of in- formation extraction. Recent works (Wang et al., 2021c; Zhong and Chen, 2021) regard these two tasks as span classification or span pair classifi- cation, and thus focus on extracting better span representations from the PLMs. ∗Corresponding author: M. Sun (sms@tsinghua.edu.cn) David assaulted a pair of restaurant workers during a night out with national squad teammates in [S]Copenhagen[/S]. [O]David[/O] assaulted a pair of restaurant workers during a night out with national squad teammates in [S]Copenhagen[/S]. David assaulted a pair of restaurant [O]workers[/O] during a night out with national squad teammates in [S]Copenhagen[/S]. David assaulted a pair of restaurant workers during a night out with national squad [O]teammates[/O] in [S]Copenhagen[/S]. Soild Marker Packed Levitated Marker [O David][/O] [O workers][/O] [O teammates][/O] Levitated Marker [S Copenhagen][/S] [O David][/O] David assaulted a pair of restaurant workers during a night out with national squad teammates in Copenhagen. [S Copenhagen][/S] [O workers][/O] [S Copenhagen][/S] [O teammates][/O] Training Process [S Copenhagen][/S] [O David][/O] [S Copenhagen][/S] [O workers][/O] [S Copenhagen][/S] [O teammates][/O] Inference Process Figure 1: An …

Content truncated to 500 words