Better Few-Shot Relation Extraction with Label Prompt Dropout

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6996 - 7006 December 7-11, 2022 ©2022 Association for Computational Linguistics Better Few-Shot Relation Extraction with Label Prompt Dropout Peiyuan Zhang and Wei Lu StatNLP Research Group Singapore University of Technology and Design peiyuan_zhang@sutd.edu.sg, luwei@sutd.edu.sg Abstract Few-shot relation extraction aims to learn to identify the relation between two entities based on very limited training examples. Recent ef- forts found that textual labels (i.e., relation names and relation descriptions) could be ex- tremely useful for learning class representa- tions, which will benefit the few-shot learn- ing task. However, what is the best way to leverage such label information in the learning process is an important research question. Ex- isting works largely assume such textual labels are always present during both learning and prediction. In this work, we argue that such ap- proaches may not always lead to optimal results. Instead, we present a novel approach called la- bel prompt dropout, which randomly removes label descriptions in the learning process. Our experiments show that our approach is able to lead to improved class representations, yield- ing significantly better results on the few-shot relation extraction task.1 1 Introduction Enabling machines to comprehend sentences and extract relations between entities has been a cru- cial task in Natural Language Processing (NLP). Conventional methods frame this task as a multi- class classification problem, trying to solve it through large-scale supervised training with LSTM (Hochreiter and Schmidhuber, 1997) or BERT (De- vlin et al., 2019) as the backbone (Zhou et al., 2016; Zhang et al., 2017; Yamada et al., 2020). Such an approach has shown great effectiveness. However, one problem left unsolved is to identify novel re- lations with only a handful of training examples. Therefore, recent studies (Han et al., 2018; Gao et al., 2019b) introduce the task of few-shot rela- tion extraction (FSRE) to study this data scarcity problem. Aligned with the success of few shot learn- ing in Computer Vision (Sung et al., 2018; Sator- 1Code available at https://github.com/jzhang38/LPD Figure 1: An example of 2-way-1-shot learning using label prompt dropout (LPD). Top: Instead of assuming textual labels are always present for support instances, LPD randomly drops out such textual labels. Here the textual label “country of origin” for the second instance is droppoed out. Bottom: LPD directly concatenates the textual label and the context sentence. The textual label serves as a prompt to guide BERT to derive a better class prototype. Note that for simplicity we use the relation names here, while in our implementation we use relation descriptions, which are lengthier and more complex. ras and Estrach, 2018), most attempts in FSRE adopt a meta learning framework (Santoro et al., 2016; Vinyals et al., 2016) that randomly samples episodes with different label sets from the training data to mimic the few shot scenario in the testing phase. As a meta learning approach, prototypi- cal network (Snell et al., 2017) aims to learn a class-agnostic metric space. A query instance is …

Content truncated to 500 words