Named Entity Recognition for Novel Types by Transfer Learning

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 899–905, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Named Entity Recognition for Novel Types by Transfer Learning Lizhen Qu1,2, Gabriela Ferraro1,2, Liyuan Zhou1, Weiwei Hou1, Timothy Baldwin1,3 1 DATA61, Australia 2 The Australian National University 3 The University of Melbourne {lizhen.qu,gabriela.ferraro,joe.zhou}@data61.csiro.au houvivid2013@gmail.com, tb@ldwin.net Abstract In named entity recognition, we often don’t have a large in-domain training corpus or a knowledge base with adequate coverage to train a model directly. In this paper, we pro- pose a method where, given training data in a related domain with similar (but not identical) named entity (NE) types and a small amount of in-domain training data, we use transfer learn- ing to learn a domain-specific NE model. That is, the novelty in the task setup is that we as- sume not just domain mismatch, but also label mismatch. 1 Introduction There are two main approaches to named entity recog- nition (NER): (i) build sequence labelling models such as conditional random fields (CRFs) (Lafferty et al., 2001) on a large manually-labelled training cor- pus (Finkel et al., 2005); and (ii) exploit knowledge bases to recognise mentions of entities in text (Rizzo and Troncy, 2012; Mendes et al., 2011). For many social media-based or security-related applications, however, we cannot assume that we will have ac- cess to either of these. An alternative is to have a small amount of in-domain training data and access to large-scale annotated data in a second domain, and perform transfer learning over both the features and label set. This is the problem setting in this paper. NER of novel named entity (NE) types poses two key challenges. First is the issue of sourcing labelled training data. Handcrafted features play a key role in supervised NER models (Turian et al., 2010), but if we have only limited training amounts of training data, we will be hampered in our ability to reliably learn feature weights. Second, the absence of target NE types in the source domain makes transfer diffi- cult, as we cannot directly apply a model trained over the source domain to the target domain. Alvarado et al. (2015) show that even if the NE label set is identical across domains, large discrepancies in the label distribution can lead to poor performance. Despite these difficulties, it is possible to transfer knowledge between domains, as related NE types often share lexical and context features. For example, the expressions give lectures and attend tutorials of- ten occur near mentions of NE types PROFESSOR and STUDENT. If only PROFESSOR is observed in the source domain but we can infer that the two classes are similar, we can leverage the training data to learn an NER model for STUDENT. In practice, differences between NE classes are often more sub- tle than this, but if we can infer, for example, that the novel NE type STUDENT aligns with NE types PERSON and UNIVERSITY, we can compose the context features of PERSON …

Content truncated to 500 words