Unsupervised Models for Named Entity Classification

Unsupervised Models for Named Entity Classification Michael Collins and Yoram Singer AT&T Labs-Research, 180 Park Avenue, Florham Park, NJ 07932 {mcollins, singer}@research, att. com Abstract This paper discusses the use of unlabeled examples for the problem of named entity classification. A large number of rules is needed for coverage of the domain, suggesting that a fairly large number of la- beled examples should be required to train a classi- fier. However, we show that the use of unlabeled data can reduce the requirements for supervision to just 7 simple "seed" rules. The approach gains leverage from natural redundancy in the data: for many named-entity instances both the spelling of the name and the context inwhich it appears are sufficient to determine its type. We present two algorithms. The first method uses a similar algorithm to that of (Yarowsky 95), with modifications motivated by (Blum and Mitchell 98). The second algorithm extends ideas from boosting algorithms, designed for supervised learning tasks, to the framework suggested by (Blum and Mitchell 98). 1 Introduction Many statistical or machine-learning approaches for natural language problems require a relatively large amount of supervision, in the form of labeled train- ing examples. Recent results (e.g., (Yarowsky 95; Brill 95; Blum and Mitchell 98)) have suggested that unlabeled data can be used quite profitably in reducing the need for supervision. This paper dis- cusses the use of unlabeled examples for the prob- lem of named entity classification. The task is to learn a function from an in- put string (proper name) to its type, which we will assume to be one of the categories Person, Organization, or Location. For example, a good classifier would identify Mrs. Frank as a per- son, Steptoe & Johnson as a company, and Hon- duras as a location. The approach uses both spelling and contextual rules. A spelling rule might be a sim- ple look-up for the string (e.g., a rule that Honduras is a location) or a rule that looks at words within a string (e.g., a rule that any string containing Mr. is a person). A contextual rule considers words sur- rounding the string in the sentence in which it ap- pears (e.g., a rule that any proper name modified by an appositive whose head is president is a person). The task can be considered to be one component of the MUC (MUC-6, 1995) named entity task (the other task is that of segmentation, i.e., pulling pos- sible people, places and locations from text before sending them to the classifier). Supervised meth- ods have been applied quite successfully to the full MUC named-entity task (Bikel et el. 97). At first glance, the problem seems quite com- plex: a large number of rules is needed to cover the domain, suggesting that a large number of labeled examples is required to train an accurate classifier. But we will show that the use of unlabeled data can drastically reduce the need for supervision. Given around 90,000 unlabeled examples, the methods de- scribed in this …

Content truncated to 500 words