Type-Aware Distantly Supervised Relation Extraction with Linked Arguments

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1891–1901, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Type-Aware Distantly Supervised Relation Extraction with Linked Arguments Mitchell Koch John Gilmer Stephen Soderland Daniel S. Weld Department of Computer Science & Engineering University of Washington Seattle, WA 98195, USA {mkoch,jgilme1,soderlan,weld}@cs.washington.edu Abstract Distant supervision has become the lead- ing method for training large-scale rela- tion extractors, with nearly universal adop- tion in recent TAC knowledge-base pop- ulation competitions. However, there are still many questions about the best way to learn such extractors. In this paper we investigate four orthogonal improvements: integrating named entity linking (NEL) and coreference resolution into argument identification for training and extraction, enforcing type constraints of linked argu- ments, and partitioning the model by rela- tion type signature. We evaluate sentential extraction perfor- mance on two datasets: the popular set of NY Times articles partially annotated by Hoffmann et al. (2011) and a new dataset, called GORECO, that is comprehensively annotated for 48 common relations. We find that using NEL for argument identi- fication boosts performance over the tra- ditional approach (named entity recogni- tion with string match), and there is further improvement from using argument types. Our best system boosts precision by 44% and recall by 70%. 1 Introduction Relation extractors are commonly trained by dis- tant supervision (also known as knowledge-based weak supervision (Hoffmann et al., 2011)), an au- tonomous technique that creates a labeled train- ing set by heuristically matching the contents of a knowledge base (KB) to mentions (substrings) in a textual corpus. For example, if a KB contained the ground tuple BornIn(Albert Einstein, Ulm) then Training Extraction KB Argument Identification Matching Train Extractor Extractor Argument Identification Figure 1: Distantly supervised extraction pipeline. a distant supervision system might label the sen- tence “While [Einstein]1 was born in [Ulm]2, he moved to Munich at an early age.” as a positive training instance of the BornIn relation. Although distant supervision is a simple idea and often cre- ates data with false positives, it has become ubiq- uitous; for example, all top-performing systems in recent TAC-KBP slot filling competitions used the method. Surprisingly, however, many aspects of distant supervision are poorly studied. In response we perform an extensive search of ways to improve distant supervision and the extraction process, in- cluding using named entity linking (NEL) and coreference to identify arguments for distant su- pervision and extraction, as well as using type con- straints and partitioning the trained model by rela- tion type signatures. The first step in the distant supervision process is argument identification (Figure 1) — finding textual mentions referring to entities that might be in some relation. Next comes matching, where KB facts, e.g. tuples such as R(e1, e2), are associated with sentences mentioning entities e1 and e2 in the assumption that many of these sentences de- scribe the relation R. Most previous systems per- form these steps by first using named entity recog- nition (NER) to identify …

Content truncated to 500 words