Label-Aware Double Transfer Learning for Cross-Specialty Medical Named Entity Recognition

Proceedings of NAACL-HLT 2018, pages 1–15 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Label-aware Double Transfer Learning for Cross-Specialty Medical Named Entity Recognition Zhenghui Wang†, Yanru Qu†, Liheng Chen†, Jian Shen†, Weinan Zhang†∗ Shaodian Zhang†‡, Yimei Gao‡, Gen Gu‡, Ken Chen‡, Yong Yu† †APEX Data and Knowledge Management Lab, Shanghai Jiao Tong University ‡Synyi LLC. {felixwzh,wnzhang,shaodian}@apex.sjtu.edu.cn chen.ken@synyi.com Abstract We study the problem of named entity recog- nition (NER) from electronic medical records, which is one of the most fundamental and critical problems for medical text mining. Medical records which are written by clini- cians from different specialties usually con- tain quite different terminologies and writ- ing styles. The difference of specialties and the cost of human annotation makes it par- ticularly difficult to train a universal medical NER system. In this paper, we propose a label- aware double transfer learning framework (La- DTL) for cross-specialty NER, so that a med- ical NER system designed for one specialty could be conveniently applied to another one with minimal annotation efforts. The trans- ferability is guaranteed by two components: (i) we propose label-aware MMD for feature representation transfer, and (ii) we perform parameter transfer with a theoretical upper bound which is also label aware. We conduct extensive experiments on 12 cross-specialty NER tasks. The experimental results demon- strate that La-DTL provides consistent accu- racy improvement over strong baselines. Be- sides, the promising experimental results on non-medical NER scenarios indicate that La- DTL is potential to be seamlessly adapted to a wide range of NER tasks. 1 Introduction The development of hospital information system and medical informatics drives the leverage of var- ious medical data for a more efficient and intel- ligent medical care service. Among many kinds of medical data, electronic health records (EHRs) are one of the most valuable and informative data as they contain detailed information about the pa- tients and the clinical practices. EHRs are essen- tial to many intelligent clinical applications, such ∗Weinan Zhang is the corresponding author. as hospital quality control and clinical decision support systems (Wu et al., 2015). Most of EHRs are recorded in an unstructured form, i.e., natural language. Hence, extracting structured informa- tion from EHRs using natural language processing (NLP), e.g., named entity recognition (NER) and entity linking, plays a fundamental role in medical informatics (Zhang and Elhadad, 2013). In this pa- per, we focus on medical NER from EHRs, which is a fundamental task and is widely studied in the research community (Nadeau and Sekine, 2007; Uzuner et al., 2011). In practice, the difficulty of building a univer- sally robust and high-performance medical NER system lies in the variety of medical terminologies and expressions among different departments of specialties and hospitals. However, building sepa- rate NER systems for so many specialties comes with a prohibitively high cost. The data privacy issue further discourages the sharing of the data across departments or hospitals, making it more difficult to train a canonical NER system to be ap- plied everywhere. …

Content truncated to 500 words