Using Gazetteers in Discriminative Information Extraction

Proceedings of the 10th Conference on Computational Natural Language Learning (CoNLL-X), pages 133–140, New York City, June 2006. c⃝2006 Association for Computational Linguistics Using Gazetteers in Discriminative Information Extraction Andrew Smith Division of Informatics University of Edinburgh United Kingdom a.p.smith-2@sms.ed.ac.uk Miles Osborne Division of Informatics University of Edinburgh United Kingdom miles@inf.ed.ac.uk Abstract Much work on information extraction has successfully used gazetteers to recognise uncommon entities that cannot be reliably identified from local context alone. Ap- proaches to such tasks often involve the use of maximum entropy-style models, where gazetteers usually appear as highly informative features in the model. Al- though such features can improve model accuracy, they can also introduce hidden negative effects. In this paper we de- scribe and analyse these effects and sug- gest ways in which they may be overcome. In particular, we show that by quarantin- ing gazetteer features and training them in a separate model, then decoding using a logarithmic opinion pool (Smith et al., 2005), we may achieve much higher accu- racy. Finally, we suggest ways in which other features with gazetteer feature-like behaviour may be identified. 1 Introduction In recent years discriminative probabilistic models have been successfully applied to a number of infor- mation extraction tasks in natural language process- ing (NLP), such as named entity recognition (NER) (McCallum and Li, 2003), noun phrase chunking (Sha and Pereira, 2003) and information extraction from research papers (Peng and McCallum, 2004). Discriminative models offer a significant advantage over their generative counterparts by allowing the specification of powerful, possibly non-independent features which would be difficult to tractably encode in a generative model. In a task such as NER, one sometimes encoun- ters an entity which is difficult to identify using lo- cal contextual cues alone because the entity has not be seen before. In these cases, a gazetteer or dic- tionary of possible entity identifiers is often useful. Such identifiers could be names of people, places, companies or other organisations. Using gazetteers one may define additional features in the model that represent the dependencies between a word’s NER label and its presence in a particular gazetteer. Such gazetteer features are often highly informative, and their inclusion in the model should in principle re- sult in higher model accuracy. However, these fea- tures can also introduce hidden negative effects tak- ing the form of labelling errors that the model makes at places where a model without the gazetteer fea- tures would have labelled correctly. Consequently, ensuring optimal usage of gazetteers can be difficult. In this paper we describe and analyse the labelling errors made by a model, and show that they gen- erally result from the model’s over-dependence on the gazetteer features for making labelling decisions. By including gazetteer features in the model we may, in some cases, transfer too much explanatory dependency to the gazetteer features from the non- gazetteer features. In order to avoid this problem, a more careful treatment of these features is required during training. We demonstrate that a traditional regularisation approach, where …

Content truncated to 500 words