Actively Supervised Clustering for Open Relation Extraction

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4985–4997 July 9-14, 2023 ©2023 Association for Computational Linguistics Actively Supervised Clustering for Open Relation Extraction Jun Zhao1∗, Yongxin Zhang1∗, Qi Zhang1†, Tao Gui2†, Zhongyu Wei3, Minlong Peng4, Mingming Sun4 1School of Computer Science, Fudan University 2Institute of Modern Languages and Linguistics, Fudan University 3School of Data Science, Fudan University 4Cognitive Computing Lab Baidu Research {zhaoj19,yongxinzhang20,qz,tgui,zywei}@fudan.edu.cn Abstract Current clustering-based Open Relation Ex- traction (OpenRE) methods usually adopt a two-stage pipeline. The first stage simul- taneously learns relation representations and assignments. The second stage manually labels several instances and thus names the relation for each cluster. However, unsupervised objectives struggle to optimize the model to derive accurate clustering assignments, and the number of clusters has to be supplied in advance. In this paper, we present a novel setting, named actively supervised clustering for OpenRE. Our insight lies in that clustering learning and relation labeling can be alternately performed, providing the necessary guidance for clustering without a significant increase in human effort. The key to the setting is selecting which instances to label. Instead of using classical active labeling strategies designed for fixed known classes, we propose a new strategy, which is applicable to dynamically discover clusters of unknown relations. Experimental results show that our method is able to discover almost all relational clusters in the data and improve the SOTA methods by 10.3% and 5.2%, on two datasets respectively. 1 Introduction Relation extraction (RE) aims to detect and extract the potential relation between the given entity pair in unstructured text. The extracted relation facts play a vital role in many downstream applications, such as knowledge base population (Ji and Grishman, 2011), search engine (Schlichtkrull et al., 2018), and question answering (Yu et al., 2017). To deal with the emerging unknown relational types in the real world, Open Relation Extraction (OpenRE) has been widely studied. The clustering-based unsupervised relation dis- covery is a classical paradigm for OpenRE (Yao *Equal Contributions. †Corresponding authors. Clustering Assignments Relation Labeling Query Label Active Supervision Relation Labeling Unsupervised two-stage OpenRE Clustering Model Clustering Model Actively Supervised OpenRE Figure 1: Compared with the existing unsupervised two-stage methods, our method can provide explicit supervision for clustering by alternately performing clustering learning and relation labeling. Note that the human effort of the two settings is comparable. et al., 2011; Marcheggiani and Titov, 2016; Elsahar et al., 2017). It can discover potential relations, by grouping several instances into relational clusters, and then manually labeling a few instances to name the relation of each cluster. Recently, Hu et al. (2020) introduced a deep clustering framework (Caron et al., 2018) into OpenRE. They iteratively cluster the relation representations that are produced by large pretrained models and use the cluster assignments as pseudo-labels to refine the representations. Unfortunately, the above unsupervised methods struggle to learn good enough representations, and the cluster assignments are error-prone. When multiple relations are mixed in a cluster, it becomes difficult to name …

Content truncated to 500 words