Semi-supervised New Event Type Induction and Description via Contrastive Loss-Enforced Batch Attention

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3805–3827 May 2-6, 2023 ©2023 Association for Computational Linguistics Semi-supervised New Event Type Induction and Description via Contrastive Loss-Enforced Batch Attention Carl Edwards and Heng Ji University of Illinois Urbana-Champaign {cne2, hengji}@illinois.edu Abstract Most event extraction methods have tradition- ally relied on an annotated set of event types. However, creating event ontologies and anno- tating supervised training data are expensive and time-consuming. Previous work has pro- posed semi-supervised approaches which lever- age seen (annotated) types to learn how to au- tomatically discover new event types. State-of- the-art methods, both semi-supervised or fully unsupervised, use a form of reconstruction loss on specific tokens in a context. In contrast, we present a novel approach to semi-supervised new event type induction using a masked con- trastive loss, which learns similarities between event mentions by enforcing an attention mech- anism over the data minibatch. We further dis- entangle the discovered clusters by approxi- mating the underlying manifolds in the data, which allows us to achieve an adjusted rand index score of 48.85%. Building on these clus- tering results, we extend our approach to two new tasks: predicting the type name of the dis- covered clusters and linking them to FrameNet frames.1 1 Introduction Discovering new event types is an important step for adapting information extraction (IE) methods to unseen domains. Existing work (Ji and Grish- man, 2008; McClosky et al., 2011; Li et al., 2013; Chen et al., 2015; Du and Cardie, 2020; Li et al., 2021a) traditionally uses a predefined list of event types and their respective annotations to learn an event extraction model. However, these annota- tions are both expensive and time-consuming to create. This problem is amplified when considering specialization-intensive domains such as scientific literature, which requires years of specialized ex- perience to understand even a specific niche. For 1The programs and resources will be publicly available at github.com/cnedwards/EventTypeBatchAttention for research purposes. example, there are a wide range of otherwise ob- scure events in biomedical literature (Krallinger et al., 2017), and better IE techniques can empower life-changing breakthroughs in these domains. To adapt IE to these specialized domains, it is critical to discover new event types automatically. There are two primary approaches in event type induction. The first is completely unsupervised induction. It includes recent neural techniques (Huang et al., 2016; Shen et al., 2021), as well as ad-hoc clustering techniques (Sekine, 2006; Cham- bers and Jurafsky, 2011) and probabilistic gener- ative methods (Cheung et al., 2013; Chambers, 2013; Nguyen et al., 2015). The second approach, semi-supervised event type induction, was recently introduced by Huang and Ji (2020). It proposes leveraging annotations for existing types to learn to discover new types; this enables taking advantage of existing resources. In this work, we pursue the second approach. Current state-of-the-art work in event type in- duction (Huang and Ji, 2020; Shen et al., 2021) uses reconstruction-based losses to find clusters of new types. Motivated by recent success …

Content truncated to 500 words