Few-shot Event Detection: An Empirical Study and a Unified View

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 11211–11236 July 9-14, 2023 ©2023 Association for Computational Linguistics Few-shot Event Detection: An Empirical Study and a Unified View Yubo Ma1, Zehao Wang2, Yixin Cao3†, Aixin Sun1† 1 S-Lab, Nanyang Technological University 2 KU Leuven 3 Singapore Management University yubo001@e.ntu.edu.sg Abstract Few-shot event detection (ED) has been widely studied, while this brings noticeable discrep- ancies, e.g., various motivations, tasks, and experimental settings, that hinder the under- standing of models for future progress. This paper presents a thorough empirical study, a unified view of ED models, and a better uni- fied baseline. For fair evaluation, we compare 12 representative methods on three datasets, which are roughly grouped into prompt-based and prototype-based models for detailed anal- ysis. Experiments consistently demonstrate that prompt-based methods, including Chat- GPT, still significantly trail prototype-based methods in terms of overall performance. To in- vestigate their superior performance, we break down their design elements along several di- mensions and build a unified framework on prototype-based methods. Under such unified view, each prototype-method can be viewed a combination of different modules from these design elements. We further combine all ad- vantageous modules and propose a simple yet effective baseline, which outperforms existing methods by a large margin (e.g., 2.7% F1 gains under low-resource setting). 1 1 Introduction Event Detection (ED) is the task of identifying event triggers and types in texts. For example, given “Cash-strapped Vivendi wants to sell Univer- sal Studios”, it is to classify the word “sell” into a TransferOwnership event. ED is a fundamental step in various tasks such as successive event-centric in- formation extraction (Huang et al., 2022; Ma et al., 2022b; Chen et al., 2022), knowledge systems (Li et al., 2020; Wen et al., 2021), story generation (Li et al., 2022a), etc. However, the annotation of event instances is costly and labor-consuming, which mo- †Corresponding Author. 1Our code will be publicly available at https://github.com/mayubo2333/fewshot_ED. Model Train Predict Train Predict Model Train Source Target (a) low-resource setting (b) class-transfer setting Model Figure 1: Task settings to access Generalization (a) and Transferability (b). Colors denote event types. tivates the research on improving ED with limited labeled samples, i.e., the few-shot ED task. Extensive studies have been carried out on few- shot ED. Nevertheless, there are noticeable discrep- ancies among existing methods from three aspects. (1) Motivation (Figure 1): Some methods focus on model’s generalization ability that learns to classify with only a few samples (Li et al., 2022b). Some other methods improve the transferability, by intro- ducing additional data, that adapts a well-trained model on the preexisting schema to a new schema using a few samples (Lu et al., 2021). There are also methods considering both abilities (Liu et al., 2020; Hsu et al., 2022). (2) Task setting: Even focusing on the same ability, methods might adopt different task settings for training and evaluation. For example, there are at least three settings for transferability: episode learning (EL, Deng et …

Content truncated to 500 words