Clustering-Based Stratified Seed Sampling for Semi-Supervised Relation Classification

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 346–355, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Clustering-based Stratified Seed Sampling for Semi-Supervised Relation Classification Longhua Qian Guodong Zhou Natural Language Processing Lab Natural Language Processing Lab School of Computer Science and Technology School of Computer Science and Technology Soochow University Soochow University 1 Shizi Street, Suzhou, China 215006 1 Shizi Street, Suzhou, China 215006 qianlonghua@suda.edu.cn gdzhou@suda.edu.cn Abstract Seed sampling is critical in semi-supervised learning. This paper proposes a clustering- based stratified seed sampling approach to semi-supervised learning. First, various clus- tering algorithms are explored to partition the unlabeled instances into different strata with each stratum represented by a center. Then, diversity-motivated intra-stratum sampling is adopted to choose the center and additional instances from each stratum to form the unla- beled seed set for an oracle to annotate. Fi- nally, the labeled seed set is fed into a bootstrapping procedure as the initial labeled data. We systematically evaluate our stratified bootstrapping approach in the semantic rela- tion classification subtask of the ACE RDC (Relation Detection and Classification) task. In particular, we compare various clustering algorithms on the stratified bootstrapping per- formance. Experimental results on the ACE RDC 2004 corpus show that our clustering- based stratified bootstrapping approach achieves the best F1-score of 75.9 on the sub- task of semantic relation classification, ap- proaching the one with golden clustering. 1 Introduction Semantic relation extraction aims to detect and classify semantic relationships between a pair of named entities occurring in a natural language text. Many machine learning approaches have been pro- posed to attack this problem, including supervised (Miller et al., 2000; Zelenko et al., 2003; Culotta and Soresen, 2004; Kambhatla, 2004; Zhao and Grishman, 2005; Zhou et al., 2005; Zhang et al., 2006; Zhou and Zhang, 2007; Zhou et al., 2007; Qian et al., 2008; Zhou et al., 2010), semi- supervised (Brin, 1998; Agichtein and Gravano, 2000; Zhang, 2004; Chen et al., 2006; Qian et al., 2009; Zhou et al., 2009), and unsupervised meth- ods (Hasegawa et al., 2004; Zhang et al., 2005; Chen et al., 2005). Current work on relation extraction mainly adopts supervised learning methods, since they achieve much better performance. However, they normally require a large number of manually la- beled relation instances, whose acquisition is both time consuming and labor intensive. In contrast, unsupervised methods do not need any manually labeled instances. Nevertheless, it is difficult to assess their performance due to the lack of evalua- tion criteria. As something between them, semi- supervised learning has received more and more attention recently. With the plenitude of unlabeled natural language text at hand, semi-supervised learning can significantly reduce the need for la- beled data with only limited sacrifice in perform- ance. For example, Abney (2002) proposes a bootstrapping algorithm which chooses the unla- beled instances with the highest probability of be- ing correctly labeled and add them in turn into the labeled training data iteratively. This paper focuses on bootstrapping-based …

Content truncated to 500 words