Data Selection in Semi-supervised Learning for Name Tagging

Proceedings of the Workshop on Information Extraction Beyond The Document, pages 48–55, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Data Selection in Semi-supervised Learning for Name Tagging Heng Ji Ralph Grishman Department of Computer Science New York University New York, NY, 10003, USA hengji@cs.nyu.edu grishman@cs.nyu.edu Abstract We present two semi-supervised learning techniques to improve a state-of-the-art multi-lingual name tagger. For English and Chinese, the overall system obtains 1.7% - 2.1% improvement in F-measure, representing a 13.5% - 17.4% relative re- duction in the spurious, missing, and in- correct tags. We also conclude that simply relying upon large corpora is not in itself sufficient: we must pay attention to unlabeled data selection too. We de- scribe effective measures to automatically select documents and sentences. 1 Introduction When applying machine learning approaches to natural language processing tasks, it is time- consuming and expensive to hand-label the large amounts of training data necessary for good per- formance. Unlabeled data can be collected in much larger quantities. Therefore, a natural ques- tion is whether we can use unlabeled data to build a more accurate learner, given the same amount of labeled data. This problem is often referred to as semi-supervised learning. It signifi- cantly reduces the effort needed to develop a training set. It has shown promise in improving the performance of many tasks such as name tag- ging (Miller et al., 2004), semantic class extrac- tion (Lin et al., 2003), chunking (Ando and Zhang, 2005), coreference resolution (Bean and Riloff, 2004) and text classification (Blum and Mitchell, 1998). However, it is not clear, when semi-supervised learning is applied to improve a learner, how the system should effectively select unlabeled data, and how the size and relevance of data impact the performance. In this paper we apply two semi-supervised learning algorithms to improve a state-of-the-art name tagger. We run the baseline name tagger on a large unlabeled corpus (bootstrapping) and the test set (self-training), and automatically generate high-confidence machine-labeled sentences as additional ‘training data’. We then iteratively re- train the model on the increased ‘training data’. We first investigated whether we can improve the system by simply using a lot of unlabeled data. By dramatically increasing the size of the corpus with unlabeled data, we did get a signifi- cant improvement compared to the baseline sys- tem. But we found that adding off-topic unlabeled data sometimes makes the performance worse. Then we tried to select relevant docu- ments from the unlabeled data in advance, and got clear further improvements. We also obtained significant improvement by self-training (boot- strapping on the test data) without any additional unlabeled data. Therefore, in contrast to the claim in (Banko and Brill, 2001), we concluded that, for some applications, effective use of large unlabeled cor- pora demands good data selection measures. We propose and quantify some effective measures to select documents and sentences in this paper. The rest of this paper is structured as follows. Section 2 briefly describes the efforts made by previous researchers to use semi-supervised …

Content truncated to 500 words