SelfORE: Self-supervised Relational Feature Learning for Open Relation Extraction

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3673–3682, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 3673 SelfORE: Self-supervised Relational Feature Learning for Open Relation Extraction Xuming Hu1, Lijie Wen1†, Yusong Xu1, Chenwei Zhang2†, Philip S. Yu1,3 1Tsinghua University, 2Amazon, 3University of Illinois at Chicago 1{hxm19,xys20}@mails.tsinghua.edu.cn 1wenlj@tsinghua.edu.cn 2cwzhang@amazon.com 3psyu@uic.edu Abstract Open relation extraction is the task of ex- tracting open-domain relation facts from nat- ural language sentences. Existing works ei- ther utilize heuristics or distant-supervised annotations to train a supervised classifier over pre-defined relations, or adopt unsuper- vised methods with additional assumptions that have less discriminative power. In this work, we propose a self-supervised frame- work named SelfORE, which exploits weak, self-supervised signals by leveraging large pretrained language model for adaptive clus- tering on contextualized relational features, and bootstraps the self-supervised signals by improving contextualized features in relation classification. Experimental results on three datasets show the effectiveness and robustness of SelfORE on open-domain Relation Extrac- tion when comparing with competitive base- lines. Source code is available1. 1 Introduction With huge amounts of information people generate, Relation Extraction (RE) aims to extract triplets of the form (subject, relation, object) from sen- tences, discovering the semantic relation that holds between two entities mentioned in the text. For example, given a sentence Derek Bell was born in Belfast, we can extract a relation BORN IN between two entities Derek Bell and Belfast. The extracted triplets from the sentence are used in various down- stream applications like web search, question an- swering, and natural language understanding. Existing RE methods work well on pre-defined relations that have already appeared either in human-annotated datasets or knowledge bases. While in practice, human annotation can be labor- intensive to obtain and hard to scale up to a large 1https://github.com/THU-BPM/SelfORE †Corresponding Authors. number of relations. Lots of efforts are made to alleviate the human annotation efforts in Relation Extraction. Distant Supervision (Mintz et al., 2009) is a widely-used method to train a supervised rela- tion extraction model with less annotation as it only requires a small amount of annotated triplets as the supervision. However, distant supervised meth- ods usually make strong assumptions on entity co- occurrence without sufficient contexts, which leads to noises and sparse matching results. More im- portantly, it works on a set of pre-defined relations, which prevents its applicability on open-domain text corpora. Open Relation Extraction (OpenRE) aims at in- ferring and extracting triplets where the target rela- tions cannot be specified in advance. Besides ap- proaches that first identify relational phrases from open-domain corpora using heuristics or external labels via distant supervision and then recognize entity pairs (Yates et al., 2007; Fader et al., 2011), clustering-based unsupervised representation learn- ing models get lots of attentions recently due to their ability to recognize triplets from meaningful semantic features with minimized or even no hu- man annotation. Yao et al. (2011) regards OpenRE as a totally unsupervised task and uses clustering method to extract triplets with new relation types. However, it …

Content truncated to 500 words