Heterogeneous Supervision for Relation Extraction: A Representation Learning Approach

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 46–56 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Heterogeneous Supervision for Relation Extraction: A Representation Learning Approach Liyuan Liu†∗ Xiang Ren†∗ Qi Zhu† Huan Gui† Shi Zhi† Heng Ji♯ Jiawei Han† † University of Illinois at Urbana-Champaign, Urbana, IL, USA ♯ Computer Science Department, Rensselaer Polytechnic Institute, USA †{ll2, xren7, qiz3, huangui2, shizhi2, hanj}@illinois.edu ♯{jih}@rpi.edu Abstract Relation extraction is a fundamental task in information extraction. Most existing methods have heavy reliance on annota- tions labeled by human experts, which are costly and time-consuming. To overcome this drawback, we propose a novel frame- work, REHESSION, to conduct relation extractor learning using annotations from heterogeneous information source, e.g., knowledge base and domain heuristics. These annotations, referred as heteroge- neous supervision, often conflict with each other, which brings a new challenge to the original relation extraction task: how to infer the true label from noisy labels for a given instance. Identifying context in- formation as the backbone of both rela- tion extraction and true label discovery, we adopt embedding techniques to learn the distributed representations of context, which bridges all components with mutual enhancement in an iterative fashion. Ex- tensive experimental results demonstrate the superiority of REHESSION over the state-of-the-art. 1 Introduction One of the most important tasks towards text un- derstanding is to detect and categorize semantic relations between two entities in a given context. For example, in Fig. 1, with regard to the sentence of c1, relation between Jesse James and Missouri should be categorized as died in. With accurate identification, relation extraction systems can pro- vide essential support for many applications. One ∗Equal contribution. example is question answering, regarding a spe- cific question, relation among entities can provide valuable information, which helps to seek better answers (Bao et al., 2014). Similarly, for medical science literature, relations like protein-protein in- teractions (Fundel et al., 2007) and gene disease associations (Chun et al., 2006) can be extracted and used in knowledge base population. Addition- ally, relation extractors can be used in ontology construction (Schutz and Buitelaar, 2005). Typically, existing methods follow the super- vised learning paradigm, and require extensive an- notations from domain experts, which are costly and time-consuming. To alleviate such drawback, attempts have been made to build relation extrac- tors with a small set of seed instances or human- crafted patterns (Nakashole et al., 2011; Carlson et al., 2010), based on which more patterns and in- stances will be iteratively generated by bootstrap learning. However, these methods often suffer from semantic drift (Mintz et al., 2009). Besides, knowledge bases like Freebase have been lever- aged to automatically generate training data and provide distant supervision (Mintz et al., 2009). Nevertheless, for many domain-specific applica- tions, distant supervision is either non-existent or insufficient (usually less than 25% of relation mentions are covered (Ren et al., 2015; Ling and Weld, 2012)). Only recently have preliminary studies been de- veloped to unite different supervisions, includ- ing knowledge bases …

Content truncated to 500 words