Unsupervised Relation Extraction from Language Models using Constrained Cloze Completion

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1263–1276 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 1263 Unsupervised Relation Extraction from Language Models using Constrained Cloze Completion Ankur Goswami, Akshata Bhat, Hadar Ohana, Theodoros Rekatsinas University of Wisconsin-Madison {agoswami6,abhat6,hohana,rekatsinas}@wisc.edu Abstract We show that state-of-the-art self-supervised language models can be readily used to ex- tract relations from a corpus without the need to train a fine-tuned extractive head. We in- troduce RE-Flex, a simple framework that per- forms constrained cloze completion over pre- trained language models to perform unsuper- vised relation extraction. RE-Flex uses contex- tual matching to ensure that language model predictions matches supporting evidence from the input corpus that is relevant to a target re- lation. We perform an extensive experimental study over multiple relation extraction bench- marks and demonstrate that RE-Flex outper- forms competing unsupervised relation extrac- tion methods based on pretrained language models by up to 27.8 F1 points compared to the next-best method. Our results show that constrained inference queries against a lan- guage model can enable accurate unsupervised relation extraction. 1 Introduction Relation extraction is a fundamental problem in constructing knowledge bases from unstructured text. The goal of relational extraction is to identify mentions of relational facts (i.e., binary relations between entities) in a text corpus. Traditionally, relation extraction systems leverage supervised ma- chine learning approaches to train specialized ex- traction models for different relations (Dong et al., 2014; Shin et al., 2015). However, advances in natural language understanding models, such as BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019), have shifted the focus towards gen- eral relation extraction where a single natural lan- guage model is used for extraction across different relations (Levy et al., 2017). A key idea behind general relation extraction is to leverage question answering (QA) models and use the reading comprehension capabilities of modern natural language models to identify relation mentions in text. For example, the re- lation drafted by can be completed for the subject Stephen Curry by answering the question Who drafted Stephen Curry? State-of- the-art results leverage fine-tuned QA models over self-supervised contextual representations (Devlin et al., 2018; Radford et al., 2018). Initial ap- proaches (Levy et al., 2017) learn extractive QA models by exploiting annotated question-answer pairs and following a supervised setting. While effective in domains related to the an- notated question-answer data, supervised extrac- tive QA approaches can fail to generalize to new domains for which annotations are not available (Dhingra et al., 2018). For this reason, more recent approaches (Lewis et al., 2019) propose to use auto- matically generated question-answer pairs for train- ing and adopt a weakly-supervised setting (Lewis et al., 2019). However, noisy or inaccurate training data leads to a significant drop in performance. In this work, we revisit the problem of general relation extraction and show that one can perform unsupervised relation extraction by directly using the generative ability of self-supervised contex- tual language models and without training a fine- tuned QA model. …

Content truncated to 500 words