Language Independent NER using a Maximum Entropy Tagger

Language Independent NER using a Maximum Entropy Tagger James R. Curran and Stephen Clark School of Informatics University of Edinburgh 2 Buccleuch Place, Edinburgh. EH8 9LW {jamesc,stephenc}@cogsci.ed.ac.uk Abstract Named Entity Recognition (NER) systems need to integrate a wide variety of information for optimal performance. This paper demonstrates that a maximum entropy tagger can effectively encode such information and identify named entities with very high accuracy. The tagger uses features which can be obtained for a vari- ety of languages and works effectively not only for English, but also for other languages such as German and Dutch. 1 Introduction Named Entity Recognition1 (NER) can be treated as a tagging problem where each word in a sentence is as- signed a label indicating whether it is part of a named entity and the entity type. Thus methods used for part of speech (POS) tagging and chunking can also be used for NER. The papers from the CoNLL-2002 shared task which used such methods (e.g. Malouf (2002), Burger et al. (2002)) reported results significantly lower than the best system (Carreras et al., 2002). However, Zhou and Su (2002) have reported state of the art results on the MUC-6 and MUC-7 data using a HMM-based tagger. Zhou and Su (2002) used a wide variety of features, which suggests that the relatively poor performance of the taggers used in CoNLL-2002 was largely due to the fea- ture sets used rather than the machine learning method. We demonstrate this to be the case by improving on the best Dutch results from CoNLL-2002 using a maximum entropy (ME) tagger. We report reasonable precision and recall (84.9 F-score) for the CoNLL-2003 English test data, and an F-score of 68.4 for the CoNLL-2003 Ger- man test data. 1We assume that NER involves assigning the correct label to an entity as well as identifying its boundaries. Incorporating a diverse set of overlapping features in a HMM-based tagger is difficult and complicates the smoothing typically used for such taggers. In contrast, a ME tagger can easily deal with diverse, overlapping fea- tures. We also use a Gaussian prior on the parameters for effective smoothing over the large feature space. 2 The ME Tagger The ME tagger is based on Ratnaparkhi (1996)’s POS tag- ger and is described in Curran and Clark (2003) . The tagger uses models of the form: p(y|x) = 1 Z(x) exp � n � i=1 λifi(x, y) � (1) where y is the tag, x is the context and the fi(x, y) are the features with associated weights λi. The probability of a tag sequence y1 . . . yn given a sentence w1 . . . wn is approximated as follows: p(y1 . . . yn|w1 . . . wn) ≈ n � i=1 p(yi|xi) (2) where xi is the context for word wi. The tagger uses beam search to find the most probable sequence given the sentence. The features are binary valued functions which pair a tag with various elements of the context; for example: …

Content truncated to 500 words