Cluster-aware Pseudo-Labeling for Supervised Open Relation Extraction

Proceedings of the 29th International Conference on Computational Linguistics, pages 1834–1841 October 12–17, 2022. 1834 Cluster-aware Pseudo-Labeling for Supervised Open Relation Extraction Bin Duan∗, Shusen Wang∗, Xingxian Liu∗, Yajing Xu Pattern Recognition & Intelligent System Laboratory, Beijing University of Posts and Telecommunications, Beijing, China {bobtuan,shusenw,liuxingxian,xyj}@bupt.edu.cn Abstract Supervised open relation extraction aims to discover novel relations by leveraging super- vised data of pre-defined relations. However, most existing methods do not achieve effective knowledge transfer from pre-defined relations to novel relations, they have difficulties gener- ating high-quality pseudo-labels for unsuper- vised data of novel relations and usually suffer from the error propagation issue. In this paper, we propose a Cluster-aware Pseudo-Labeling (CaPL) method to improve the pseudo-labels quality and transfer more knowledge for discov- ering novel relations. Specifically, the model is first pre-trained with the pre-defined relations to learn the relation representations. To im- prove the pseudo-labels quality, the distances between each instance and all cluster centers are used to generate cluster-aware soft pseudo- labels for novel relations. To mitigate the catas- trophic forgetting issue, we design the consis- tency regularization loss to make better use of the pseudo-labels and jointly train the model with both unsupervised and supervised data. Experimental results on two public datasets demonstrate that our proposed method achieves new state-of-the-arts performance1. 1 Introduction Open relation extraction (OpenRE) has been pro- posed to extract the novel relations that are not defined or observed beforehand. Previous meth- ods can be classified into two types: unsupervised and supervised. Unsupervised OpenRE (Yao et al., 2011, 2012; Marcheggiani and Titov, 2016; Elsa- har et al., 2017; Tran et al., 2020; Hu et al., 2020) regards the OpenRE as a totally unsupervised task which first extracts the feature and then clusters them. However, these methods don’t take full ad- vantage of the large amounts of existing relational ∗The first three authors contribute equally. Yajing Xu is the corresponding author. 1Code is available at https://github.com/BobTuan/CaPL Hard Pseudo Label 1s 2s 3s Soft Pseudo Label 1s 2s 3s Henri was born in Paris 2s: Tom was born in 2008 : 3s : 1s Tom was born in Chicago Figure 1: Relation instance S1 and S3 belong to the same relation type while S2 with similar context is from another relation type. However, the hard pseudo-labels produced by common clustering methods contain much noise which causes the error propagation issue. We generate soft pseudo-labels that contain the information about true cluster to improve the pseudo-labels quality. facts in knowledge bases. Hence, the supervised OpenRE is proposed which leverages the super- vised data of pre-defined relations to guide the learning of the unsupervised data of novel rela- tions. In this paper, we focus on the latter setting, supervised OpenRE. Since the classes between pre-defined relations and novel relations are disjoint, the main challenge of supervised OpenRE is how to make the best use of the prior knowledge in pre-defined relations to extract novel relations. Wu et al. (2019) pro- poses relational siamese networks to transfer the knowledge from …

Content truncated to 500 words