Pre-training to Match for Unified Low-shot Relation Extraction

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 5785 - 5795 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Pre-training to Match for Unified Low-shot Relation Extraction Fangchao Liu1,3, Hongyu Lin1,∗, Xianpei Han1,2,4, Boxi Cao1,3, Le Sun1,2 1Chinese Information Processing Laboratory 2State Key Laboratory of Computer Science Institute of Software, Chinese Academy of Sciences, Beijing, China 3University of Chinese Academy of Sciences, Beijing, China 4Beijing Academy of Artificial Intelligence, Beijing, China {fangchao2017,hongyu,xianpei,boxi2020,sunle}@iscas.ac.cn Abstract Low-shot relation extraction (RE) aims to rec- ognize novel relations with very few or even no samples, which is critical in real scenario ap- plication. Few-shot and zero-shot RE are two representative low-shot RE tasks, which seem to be with similar target but require totally dif- ferent underlying abilities. In this paper, we propose Multi-Choice Matching Networks to unify low-shot relation extraction. To fill in the gap between zero-shot and few-shot RE, we propose the triplet-paraphrase meta-training, which leverages triplet paraphrase to pre-train zero-shot label matching ability and uses meta- learning paradigm to learn few-shot instance summarizing ability. Experimental results on three different low-shot RE tasks show that the proposed method outperforms strong baselines by a large margin, and achieve the best perfor- mance on few-shot RE leaderboard1. 1 Introduction Relation extraction (RE) aims to extract the re- lation between two given entities in the context. The most popular approaches to build RE mod- els are based on supervised learning (Zeng et al., 2014; Baldini Soares et al., 2019). Despite the su- perior performance, supervised relation extraction approaches severely suffer from the data bottleneck, which restricts their application to more relation types in real scenarios. Consequently, low-shot relation extraction has become a recent research hotspot in RE area. There are two mainstream learning paradigms widely ex- plored in low-shot relation extraction, namely zero- shot RE (Levy et al., 2017) and few-shot RE (Han et al., 2018). Few-shot relation extraction aims to identify instances of novel relation type with only a few illustrative instances, while zero-shot RE is more progressive, which only uses external ∗Corresponding authors. 1https://thunlp.github.io/2/fewrel2_nota.html Label Instance Relation Semantics Similarity Sup. Ins. Relation Semantics Instance Similarity (a) Zero-Shot RE (b) Few-Shot RE Figure 1: Difference between zero-shot RE and few- shot RE. (a) Zero-shot requires for the ability of label semantic matching, while (b) Few-shot requires for the ability of support instance (Sup. Ins.) summarizing. knowledge and the name or definition of the novel relations to recognize them. Because low-shot RE only requires very limited manually annotated data, it can effectively alleviate data bottlenecks in con- ventional RE and therefore attached great attention. However, even with similar goals, zero-shot RE and few-shot RE actually require different funda- mental abilities. Specifically, zero-shot RE is built on label semantic matching ability, which requires models to sufficiently exploit the label semantic of given novel relations, and matches relations and queries based on their underlying semantics. While few-shot RE is built on instance semantic summa- rizing ability, which requires a model …

Content truncated to 500 words