Semi-supervised Relation Extraction with Large-scale Word Clustering

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 521–529, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Semi-supervised Relation Extraction with Large-scale Word Clustering Ang Sun Ralph Grishman Satoshi Sekine Computer Science Department New York University {asun,grishman,sekine}@cs.nyu.edu Abstract We present a simple semi-supervised relation extraction system with large-scale word clustering. We focus on systematically exploring the effectiveness of different cluster-based features. We also propose several statistical methods for selecting clusters at an appropriate level of granularity. When training on different sizes of data, our semi-supervised approach consistently outperformed a state-of-the-art supervised baseline system. 1 Introduction Relation extraction is an important information extraction task in natural language processing (NLP), with many practical applications. The goal of relation extraction is to detect and characterize semantic relations between pairs of entities in text. For example, a relation extraction system needs to be able to extract an Employment relation between the entities US soldier and US in the phrase US soldier. Current supervised approaches for tackling this problem, in general, fall into two categories: feature based and kernel based. Given an entity pair and a sentence containing the pair, both approaches usually start with multiple level analyses of the sentence such as tokenization, partial or full syntactic parsing, and dependency parsing. Then the feature based method explicitly extracts a variety of lexical, syntactic and semantic features for statistical learning, either generative or discriminative (Miller et al., 2000; Kambhatla, 2004; Boschee et al., 2005; Grishman et al., 2005; Zhou et al., 2005; Jiang and Zhai, 2007). In contrast, the kernel based method does not explicitly extract features; it designs kernel functions over the structured sentence representations (sequence, dependency or parse tree) to capture the similarities between different relation instances (Zelenko et al., 2003; Bunescu and Mooney, 2005a; Bunescu and Mooney, 2005b; Zhao and Grishman, 2005; Zhang et al., 2006; Zhou et al., 2007; Qian et al., 2008). Both lines of work depend on effective features, either explicitly or implicitly. The performance of a supervised relation extraction system is usually degraded by the sparsity of lexical features. For example, unless the example US soldier has previously been seen in the training data, it would be difficult for both the feature based and the kernel based systems to detect whether there is an Employment relation or not. Because the syntactic feature of the phrase US soldier is simply a noun-noun compound which is quite general, the words in it are crucial for extracting the relation. This motivates our work to use word clusters as additional features for relation extraction. The assumption is that even if the word soldier may never have been seen in the annotated Employment relation instances, other words which share the same cluster membership with soldier such as president and ambassador may have been observed in the Employment instances. The absence of lexical features can be compensated by 521 the cluster features. Moreover, word clusters may implicitly correspond to different relation classes. For example, the cluster of …

Content truncated to 500 words