ICA-Proto: Iterative Cross Alignment Prototypical Network for Incremental Few-Shot Relation Classification

Findings of the Association for Computational Linguistics: EACL 2023, pages 2275–2284 May 2-6, 2023 ©2023 Association for Computational Linguistics ICA-Proto: Iterative Cross Alignment Prototypical Network for Incremental Few-Shot Relation Classification Wangjie Jiang1,∗, Zhihao Ye2,∗, Bang Liu3, Ruihui Zhao2 Jianguang Zheng2, Mengyao Li4, Zhiyong Li4, Yujiu Yang1,†, Yefeng Zheng2,† 1Tsinghua Shenzhen International Graduate School, Tsinghua University 2Tencent Jarvis Lab, 3Université de Montréal Mila & CIFAR, 4Hunan University jwj20@mails.tsinghua.edu.cn,evanzhye@tencent.com yang.yujiu@sz.tsinghua.edu.cn,yefengzheng@tencent.com Abstract In the task of incremental few-shot relation clas- sification, model performance is always limited by the incompatibility between the base feature embedding space and the novel feature embed- ding space. To tackle the issue, we propose a novel model named ICA-Proto: Iterative Cross Alignment prototypical network. Specifically, we incorporate the query representation into the encoding of novel prototypes and utilize the query-aware prototypes to update the query representation at the same time. Further, we im- plement the above process iteratively to achieve more interaction. In addition, a novel prototype quadruplet loss is designed to regulate the spa- tial distributions of embedding space, so as to make it easier for the relation classification. Ex- perimental results on two benchmark datasets demonstrate that ICA-Proto significantly out- performs the state-of-the-art baseline model. 1 Introduction Relation classification (RC) is an important sub- task of relation extraction (RE), aims at classifying the relation between two marked entities in a given sentence. For example, the instance “[Newton]e1 served as the president of [the Royal Society]e2" expresses the relation member_of between the two entities Newton and the Royal Society. Some con- ventional methods (Zeng et al., 2014; Gormley et al., 2015; Soares et al., 2019) for relation classifi- cation adopt supervised training and usually suffer from the scarcity of manually annotated data. To alleviate this problem, distant supervision (DS) is adopted to automatically label abundant training in- stances by heuristically aligning knowledge graphs (KGs) with texts (Mintz et al., 2009). However, existing DS-based methods fail to deal with the problem of long-tail relations in KGs and still suf- fer from data deficiency (Han et al., 2018). ∗ Equal contribution. † Corresponding authors. base1-P86 base2-P403 base3-P27 novel1-P39 novel2-P800 novel3-P466 Figure 1: Visualization of the representations of the query instances and prototypes of BERT-IncreProto. We randomly sampled three base relations and three novel relations from the real-world dataset FewRel 1.0, each relation with its corresponding prototype (triangles for base relations and stars for novel relations) and eight query instances (points). To address the above long-tail problem, few-shot RC was proposed, which formulates RC in a few- shot learning scenario. This task requires the mod- els trained with base relations to generalize well to novel relations with only few labeled instances. Base relations are those relations that contain ad- equate instances and can be utilized effectively in the training phase to mimic the test phase on novel relations with few samples. Fine-tuning pre-trained models (Bengio, 2012; Gao et al., 2020) is straight- forward while suffering from the overfitting prob- lem. Therefore, metric-based methods (Ravi and Larochelle, 2017; Dong et al., 2020; …

Content truncated to 500 words