NITE: A Neural Inductive Teaching Framework for Domain Specific NER

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2652–2657 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics NITE: A Neural Inductive Teaching Framework for Domain-Specific NER Siliang Tang, Ning Zhang, Jinjian Zhang, Fei Wu and Yueting Zhuang College of Computer Science, Zhejiang University, China {siliang, aning, jibjianzhang, wufei, yzhuang}@zju.edu.cn Abstract In domain-specific NER, due to insuffi- cient labeled training data, deep models usually fail to behave normally. In this pa- per, we proposed a novel Neural Inductive TEaching framework (NITE) to transfer knowledge from existing domain-specific NER models into an arbitrary deep neu- ral network in a teacher-student training manner. NITE is a general framework that builds upon transfer learning and multiple instance learning, which collaboratively not only transfers knowledge to a deep stu- dent network but also reduces the noise from teachers. NITE can help deep learn- ing methods to effectively utilize existing resources (i.e., models, labeled and unla- beled data) in a small domain. The experi- ment resulted on Disease NER proved that without using any labeled data, NITE can significantly boost the performance of a CNN-bidirectional LSTM-CRF NER neu- ral network nearly over 30% in terms of F1-score. 1 Introduction Domain-specific Named Entity Recognition (DNER), which aims to identify domain specific entity mentions and their categories, plays an important role in domain document classification, retrieval and content analysis. It is also a foun- dation for further level of complex information extraction tasks, serves as cornerstone in the knowledge computing process of transforming data into machine readable knowledge (Zhuang et al., 2017). Domain-specific NER is a challeng- ing problem. For example, in biomedical domain, the number of unseen biomedical entity mentions (such as disease names, chemical names), their abbreviations or acronyms, as well as multiple names of the same entity is growing fast with the rapid increase of biomedical literatures and clinical records. However, the performance of a learning based NER system relies heavily on data annotation, which is quite expensive. The situation is even worse in domain-specific NER systems, since their data annotation requires the engage of domain experts. Therefore, in many special domains, only trained models or APIs are available, while their training data are private and inaccessible. On the other hand, due to insufficient labeled training data, deep models usually fail to behave normally in such domain, and state-of-the-art methods in these domains are usually dominated by rule based deductive methods or shallow model with hand-crafted features. However, the way of pre-defining useful domain specific hand-crafted features or rules are usually unavailable to the public. In this paper, we proposed a novel Neural In- ductive TEaching framework (NITE) to transfer knowledge from existing models into an arbitrary deep neural network. The idea of NITE is mainly borrowed from Transfer learning (Pan and Yang, 2010) where previously learned knowledge can aid current situation and solve problems with bet- ter solutions. In NITE, existing NER models be- have like inefficient teachers to teach a deep neu- ral network (we called …

Content truncated to 500 words