Exploiting Feature Hierarchy for Transfer Learning in Named Entity Recognition

Proceedings of ACL-08: HLT, pages 245–253, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Exploiting Feature Hierarchy for Transfer Learning in Named Entity Recognition Andrew Arnold, Ramesh Nallapati and William W. Cohen Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA, USA {aarnold, nmramesh, wcohen}@cs.cmu.edu Abstract We present a novel hierarchical prior struc- ture for supervised transfer learning in named entity recognition, motivated by the common structure of feature spaces for this task across natural language data sets. The problem of transfer learning, where information gained in one learning task is used to improve perfor- mance in another related task, is an important new area of research. In the subproblem of do- main adaptation, a model trained over a source domain is generalized to perform well on a re- lated target domain, where the two domains’ data are distributed similarly, but not identi- cally. We introduce the concept of groups of closely-related domains, called genres, and show how inter-genre adaptation is related to domain adaptation. We also examine multi- task learning, where two domains may be re- lated, but where the concept to be learned in each case is distinct. We show that our prior conveys useful information across domains, genres and tasks, while remaining robust to spurious signals not related to the target do- main and concept. We further show that our model generalizes a class of similar hierarchi- cal priors, smoothed to varying degrees, and lay the groundwork for future exploration in this area. 1 Introduction 1.1 Problem definition Consider the task of named entity recognition (NER). Specifically, you are given a corpus of news articles in which all tokens have been labeled as ei- ther belonging to personal name mentions or not. The standard supervised machine learning problem is to learn a classifier over this training data that will successfully label unseen test data drawn from the same distribution as the training data, where “same distribution” could mean anything from having the train and test articles written by the same author to having them written in the same language. Having successfully trained a named entity classifier on this news data, now consider the problem of learning to classify tokens as names in e-mail data. An intuitive solution might be to simply retrain the classifier, de novo, on the e-mail data. Practically, however, large, labeled datasets are often expensive to build and this solution would not scale across a large number of different datasets. Clearly the problems of identifying names in news articles and e-mails are closely related, and learning to do well on one should help your per- formance on the other. At the same time, however, there are serious differences between the two prob- lems that need to be addressed. For instance, cap- italization, which will certainly be a useful feature in the news problem, may prove less informative in the e-mail data since the rules of capitalization are followed less strictly in that domain. These are the problems we address in this paper. In …

Content truncated to 500 words