Shorten the Long Tail for Rare Entity and Event Extraction

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1339–1350 May 2-6, 2023 ©2023 Association for Computational Linguistics Shorten the Long Tail for Rare Entity and Event Extraction Pengfei Yu, Heng Ji University of Illinois Urbana-Champaign {pengfei4,hengji}@illinois.edu Abstract The distribution of knowledge elements such as entity types and event types is long-tailed in natural language. Hence information extrac- tion datasets naturally conform a long-tailed distribution. Although imbalanced datasets can teach the model about the useful real-world bias, deep learning models may learn features not generalizable to rare or unseen mentions of entities or events during evaluation, espe- cially for rare types without sufficient training instances. Existing approaches for the long- tailed learning problem seek to manipulate the training data by re-balancing, augmentation or introducing extra prior knowledge. In compar- ison, we propose to handle the generalization challenge by making the evaluation instances closer to the frequent training cases. We de- sign a new transformation module that trans- forms infrequent candidate mention representa- tion during evaluation with the average mention representation in the training dataset. Experi- mental results on classic benchmarks on three entity or event extraction datasets demonstrate the effectiveness of our framework. 1 1 Introduction Long-tailed distributions are common in natu- ral language processing tasks. This natural phe- nomenon of long-tailed distribution is formulated as Zipf’s Law (Reed, 2001). For information ex- traction, knowledge elements such as entities, re- lations and events typically conform a long-tailed distribution in natural language. This leads to the imbalanced distribution of types in the benchmark datasets unless manual manipulation is performed to balance the dataset. We show the entity type distribution of an entity extraction dataset (Few- NERD (Ding et al., 2021)), and the event type dis- tribution of two event extraction datasets (ACE 2005 (Walker et al., 2006) and MAVEN (Wang 1Code is available at https://github.com/ Perfec-Yu/LongTailIE et al., 2020)) in Figure 1. All three datasets have similar long-tailed distributions. Among them, MAVEN and Few-NERD are two relatively large- scale datasets, but the training mentions are still concentrated on a small number of frequent types and majority of the entity/event types are rare types without sufficient training examples. Long-tailed distribution is a generic problem not limited to NLP research. In fact, a lot of previ- ous work on the long-tailed learning is in the com- puter vision domain (Lin et al., 2017; Kang et al., 2020). These approaches consider the imbalance in the type distribution as the major problem and are mostly based on balancing the datasets by up- weighting the rare types and downweighting the frequent types. However, we argue that this line of research faces significant challenges when applying to the information extraction task. First, balancing the dataset breaks the real-world long-tailed prior in the datasets and may lead the model to mistak- enly predict the long-tailed types too often. We ob- serve this phenomenon in our experiments in Sec- tion 3 especially for the Classifier Re-training base- line (Kang et al., 2020). …

Content truncated to 500 words