Collective Cross-Document Relation Extraction Without Labelled Data

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1013–1023, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Collective Cross-Document Relation Extraction Without Labelled Data Limin Yao Sebastian Riedel Andrew McCallum University of Massachusetts, Amherst {lmyao,riedel,mccallum}@cs.umass.edu Abstract We present a novel approach to relation ex- traction that integrates information across doc- uments, performs global inference and re- quires no labelled text. In particular, we tackle relation extraction and entity identifi- cation jointly. We use distant supervision to train a factor graph model for relation ex- traction based on an existing knowledge base (Freebase, derived in parts from Wikipedia). For inference we run an efficient Gibbs sam- pler that leads to linear time joint inference. We evaluate our approach both for an in- domain (Wikipedia) and a more realistic out- of-domain (New York Times Corpus) setting. For the in-domain setting, our joint model leads to 4% higher precision than an isolated local approach, but has no advantage over a pipeline. For the out-of-domain data, we ben- efit strongly from joint modelling, and observe improvements in precision of 13% over the pipeline, and 15% over the isolated baseline. 1 Introduction Relation Extraction is the task of predicting seman- tic relations over entities expressed in structured or semi-structured text. This includes, for example, the extraction of employer-employee relations men- tioned in newswire, or protein-protein interactions expressed in biomedical papers. It also includes the prediction of entity types such as country, citytown or person, if we consider entity types as unary rela- tions. A particularly attractive approach to relation ex- traction is based on distant supervision.1 Here in 1Also called self training, or weak supervision. place of annotated text, only an existing knowl- edge base (KB) is needed to train a relation extrac- tor (Mintz et al., 2009; Bunescu and Mooney, 2007; Riedel et al., 2010). The facts in the KB are heuris- tically aligned to an unlabelled training corpus, and the resulting alignment is the basis for learning the extractor. Naturally, the predictions of a distantly supervised relation extractor will be less accurate than those of a supervised one. While facts of existing knowledge bases are inexpensive to come by, the heuristic align- ment to text will often lead to noisy patterns in learn- ing. When applied to unseen text, these patterns will produce noisy facts. Indeed, we find that extraction precision still leaves much room for improvement. This room is not as large as in previous work (Mintz et al., 2009) where target text and training KB are closely related. However, when we use the knowl- edge base Freebase (Bollacker et al., 2008) and the New York Times corpus (Sandhaus, 2008), we ob- serve very low precision. For example, the preci- sion of the top-ranked 50 nationality relation instances is only 28%. On inspection, it turns out that many of the errors can be easily identified: they amount to violations of basic compatibility constraints between facts. In particular, we observe unsatisfied selectional pref- erences of …

Content truncated to 500 words