Teaching a Weaker Classifier: Named Entity Recognition on Upper Case Text

Teaching a Weaker Classifier: Named Entity Recognition on Upper Case Text Hai Leong Chieu DSO National Laboratories 20 Science Park Drive Singapore 118230 chaileon@dso.org.sg Hwee Tou Ng Department of Computer Science School of Computing National University of Singapore 3 Science Drive 2 Singapore 117543 nght@comp.nus.edu.sg Abstract This paper describes how a machine- learning named entity recognizer (NER) on upper case text can be improved by us- ing a mixed case NER and some unlabeled text. The mixed case NER can be used to tag some unlabeled mixed case text, which are then used as additional training mate- rial for the upper case NER. We show that this approach reduces the performance gap between the mixed case NER and the upper case NER substantially, by 39% for MUC-6 and 22% for MUC-7 named en- tity test data. Our method is thus useful in improving the accuracy of NERs on up- per case text, such as transcribed text from automatic speech recognizers where case information is missing. 1 Introduction In this paper, we propose using a mixed case named entity recognizer (NER) that is trained on labeled text, to further train an upper case NER. In the Sixth and Seventh Message Understanding Confer- ences (MUC-6, 1995; MUC-7, 1998), the named entity task consists of labeling named entities with the classes PERSON, ORGANIZATION, LOCA- TION, DATE, TIME, MONEY, and PERCENT. We conducted experiments on upper case named entity recognition, and showed how unlabeled mixed case text can be used to improve the results of an up- per case NER on the official MUC-6 and MUC-7 Mixed Case: Consuela Washington, a longtime House staffer and an expert in securities laws, is a leading candidate to be chairwoman of the Securities and Exchange Commission in the Clinton administration. Upper Case: CONSUELA WASHINGTON, A LONGTIME HOUSE STAFFER AND AN EX- PERT IN SECURITIES LAWS, IS A LEADING CANDIDATE TO BE CHAIRWOMAN OF THE SECURITIES AND EXCHANGE COMMIS- SION IN THE CLINTON ADMINISTRATION. Figure 1: Examples of mixed and upper case text test data. Besides upper case text, this approach can also be applied on transcribed text from auto- matic speech recognizers in Speech Normalized Or- thographic Representation (SNOR) format, or from optical character recognition (OCR) output. For the English language, a word starting with a capital let- ter often designates a named entity. Upper case NERs do not have case information to help them to distinguish named entities from non-named en- tities. When data is sparse, many named entities in the test data would be unknown words. This makes upper case named entity recognition more difficult than mixed case. Even a human would experience greater difficulty in annotating upper case text than mixed case text (Figure 1). We propose using a mixed case NER to “teach” an upper case NER, by making use of unlabeled mixed case text. With the abundance of mixed case un- Computational Linguistics (ACL), Philadelphia, July 2002, pp. 481-488. Proceedings of the 40th Annual Meeting of the Association for labeled texts available …

Content truncated to 500 words