Learning to Progressively Recognize New Named Entities with Sequence to Sequence Models

Proceedings of the 27th International Conference on Computational Linguistics, pages 2181–2191 Santa Fe, New Mexico, USA, August 20-26, 2018. 2181 Learning to Progressively Recognize New Named Entities with Sequence to Sequence Models Lingzhen Chen University of Trento Povo, Italy lingzhen.chen@unitn.it Alessandro Moschitti∗ Amazon Manhattan Beach, CA, USA amosch@amazon.com Abstract In this paper, we propose to use a sequence to sequence model for Named Entity Recognition (NER) and we explore the effectiveness of such model in a progressive NER setting – a Transfer Learning (TL) setting. We train an initial model on source data and transfer it to a model that can recognize new NE categories in the target data during a subsequent step, when the source data is no longer available. Our solution consists in: (i) to reshape and re-parametrize the output layer of the first learned model to enable the recognition of new NEs; (ii) to leave the rest of the architecture unchanged, such that it is initialized with parameters transferred from the initial model; and (iii) to fine tune the network on the target data. Most importantly, we design a new NER approach based on sequence to sequence (Seq2Seq) models, which can intuitively work better in our progressive setting. We compare our approach with a Bidirectional LSTM, which is a strong neural NER model. Our experiments show that the Seq2Seq model performs very well on the standard NER setting and it is more robust in the progressive setting. Our approach can recognize previously unseen NE categories while preserving the knowledge of the seen data. 1 Introduction Standard NER models are trained and tested on data with the same NE label set. In this paper, we explore a progressive setting, where (i) in the initial step, the model is trained on a dataset DS with certain NE categories; and (ii) in the subsequent step, we further train the model on a dataset DT containing all categories from DS along with the new ones, without using the examples from DS since we assume they are no longer available. Our goal is to learn to recognize the new NE categories in DT while keeping the knowledge of previously seen categories. This study is motivated by the application of NER model in industry scenarios, where different com- panies may aim to recognize different types of NEs in the text. For example, users interested in finance would probably target entities such as companies or banks while other users interested in politics would target entities such as senators, bills, ministries, etc. Nevertheless, there are certain types of commonly- seen NE categories such as locations or dates that most users would like to recognize. These common NEs are typically provided with more available labeled data. Hence, methods that can utilize such data in new domains are rather useful. Additionally, the possibility to only use the models trained on such data is very important as it avoids the NER designer to deliver most training data to the user for adaptation in new domains. The setting above …

Content truncated to 500 words