HiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extraction

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5970 - 5980 July 10-15, 2022 ©2022 Association for Computational Linguistics HiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extraction Shuliang Liu1∗, Xuming Hu1∗, Chenwei Zhang2†, Shu’ang Li1, Lijie Wen1†, Philip S. Yu1,3 1Tsinghua University 2Amazon 3University of Illinois at Chicago {liusl19,hxm19,lisa18}@mails.tsinghua.edu.cn cwzhang@amazon.com; wenlj@tsinghua.edu.cn; psyu@uic.edu Abstract Unsupervised relation extraction aims to ex- tract the relationship between entities from natural language sentences without prior in- formation on relational scope or distribution. Existing works either utilize self-supervised schemes to refine relational feature signals by iteratively leveraging adaptive clustering and classification that provoke gradual drift prob- lems, or adopt instance-wise contrastive learn- ing which unreasonably pushes apart those sen- tence pairs that are semantically similar. To overcome these defects, we propose a novel contrastive learning framework named HiURE, which has the capability to derive hierarchi- cal signals from relational feature space using cross hierarchy attention and effectively opti- mize relation representation of sentences un- der exemplar-wise contrastive learning. Exper- imental results on two public datasets demon- strate the advanced effectiveness and robust- ness of HiURE on unsupervised relation ex- traction when compared with state-of-the-art models. Source code is available here1. 1 Introduction Relation Extraction (RE) aims to discover the se- mantic (binary) relation that holds between two entities from plain text. For instance, “Kisselhead was born in Adriantail ...", we can extract a re- lation /people/person/place_of_birth between the two head-tail entities. The extracted re- lations could be used in various downstream appli- cations such as information retrieval (Corcoglioniti et al., 2016), question answering (Bordes et al., 2014), and dialog systems (Madotto et al., 2018). Existing RE methods can achieve decent results with manually annotated data or human-curated knowledge bases. While in practice, human an- notation can be labor-intensive to obtain and hard ∗Equal contribution. †Corresponding authors. 1https://github.com/THU-BPM/HiURE to scale up to newly created relations. Lots of ef- forts are devoted to alleviating the impact of human annotations in relation extraction. Unsupervised Relation Extraction (URE) is especially promising since it does not require any prior information on relation scope and distribution. The main challenge in URE is how to cluster semantic information of sentences in the relational feature space. Simon et al. (2019) adopted skewness and dis- persion losses to enforce the relation classifier to be confident in the relational feature prediction and ensure all relation types can be predicted aver- agely in a minibatch. But it still requires the exact number of relation types in advance, and the rela- tion classifier could not be improved by obtained clustering results. Hu et al. (2020) encoded rela- tional feature space in a self-supervised method that bootstraps relational feature signals by leverag- ing adaptive clustering and classification iteratively. Nonetheless, like other self-training methods, the noisy clustering results will iteratively result in the model deviating from the global minima, which is also known as the gradual drift problem (Curran et al., 2007; Zhang et al., 2021a). Peng …

Content truncated to 500 words