Focused training sets to reduce noise in NER feature models

Proceedings of NAACL-HLT 2013, pages 411–415, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Focused training sets to reduce noise in NER feature models Amber McKenzie Computer Science and Engineering Department University of South Carolina mckenzie.amber@gmail.com Abstract Feature and context aggregation play a large role in current NER systems, allowing significant opportunities for research into op- timizing these features to cater to different domains. This work strives to reduce the noise introduced into aggregated features from dis- parate and generic training data in order to al- low for contextual features that more closely model the entities in the target data. The pro- posed approach trains models based on only a part of the training set that is more similar to the target domain. To this end, models are trained for an existing NER system using the top documents from the training set that are similar to the target document in order to demonstrate that this technique can be applied to improve any pre-built NER system. Initial results show an improvement over the Univer- sity of Illinois NE tagger with a weighted av- erage F1 score of 91.67 compared to the Illinois tagger’s score of 91.32. This research serves as a proof-of-concept for future planned work to cluster the training docu- ments to produce a number of more focused models from a given training set, thereby re- ducing noise and extracting a more repre- sentative feature set. 1 Introduction Though research in the area of named entity recognition (NER) is fairly extensive, current state- of-the-art solutions are generic, succeeding only for domains similar to their training data, and still fail to adequately provide functionality that is adaptable to a broad range of domains (Tkachenko and Simanovsky, 2012). This leaves room for im- provement in designing a system that can more easily adapt to previously unseen data. In particu- lar, the increasingly popular feature set produced by feature and context aggregation provides many opportunities for different types of optimization given the strong correlation between the training input and the feature values that are produced. This is due to the fact that aggregation looks at features at a document or corpus level, rather than at the token level, and therefore will be sensitive to changes in the training set. This research looks to exploit this aspect of feature and context aggrega- tion by identifying portions of a training set that are more similar to the target data and will thus provide feature values that are likely more repre- sentative of the entities within that data. Rather than train a model with a full training set, this approach extracts portions of the training data that are most similar to the target data and trains a model using only those documents. This initial work tailors a model to a given target docu- ment to demonstrate that less, but more appropri- ate, training data is preferable to a full generic training set. Similar to that of Dalton et al. (2011), in which they use …

Content truncated to 500 words