Neural Entity Recognition with Gazetteer based Fusion

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3291–3295 August 1–6, 2021. ©2021 Association for Computational Linguistics 3291 Neural Entity Recognition with Gazetteer based Fusion Qing Sun Amazon AWS AI qinsun@amazon.com Parminder Bhatia Amazon AWS AI parmib@amazon.com Abstract Incorporating external knowledge into Named Entity Recognition (NER) systems has been widely studied in the generic domain. In this paper, we focus on clinical domain where only limited data is accessible and interpretability is important. Recent advancement in technol- ogy and the acceleration of clinical trials has resulted in the discovery of new drugs, pro- cedures as well as medical conditions. These factors motivate towards building robust zero- shot NER systems which can quickly adapt to new medical terminology. We propose an auxiliary gazetteer model and fuse it with an NER system, which results in better ro- bustness and interpretability across different clinical datasets. Our gazetteer based fusion model is data efficient, achieving +1.7 micro- F1 gains on the i2b2 dataset using 20% train- ing data, and brings + 4.7 micro-F1 gains on novel entity mentions never presented during training. Moreover, our fusion model is able to quickly adapt to new mentions in gazetteers without re-training and the gains from the pro- posed fusion model are transferable to related datasets. 1 Introduction Named entity recognition (NER) (Lample et al., 2016; Ma and Hovy, 2016) aims to identify text mentions of specific entity types. In clinical do- mains, it’s particularly useful for automatic infor- mation extraction, e.g., diagnosis information and adverse drug events, which could be applied for a variety of downstream tasks such as clinical event surveillance, decision support (Jin et al., 2018), pharmacovigilance, and drug efficacy studies. We have witnessed a rapid progress on NER models using deep neural networks. However, ap- plying them to clinical domain (Bhatia et al., 2019) is hard due to the following challenges: (a) accessi- bility of limited data, (b) discovery of new drugs, procedures and medical conditions and the (c) need for building interpretable and explainable models. Motivated by these, we attempt to incorporate ex- ternal name or ontology knowledge, e.g., Remde- sivir is a DRUG and COVID-19 is a Medical Condition, into neural NER models for clini- cal applications. Recent work on leveraging external knowledge can be categorized into two categories - Gazetteer embedding and Gazetteer models. Recent work has primarily focused on gazetteer embeddings. Song et al. (2020) feed the concatenation of BERT out- put and gazetteer embedding into Bi-LSTM-CRF. Peshterliev et al. (2020) use self-attention over gazetteer types to enhance gazetteer embedding and then concatenate it with ELMO, char CNN and GloVe embeddings. By contrast, the basic idea of gazetteer model is to treat ontology knowledge as a new clinical modality. Magnolini et al. (2019) combine outputs of Bi-LSTM and gazetteer model and feed them into CRF layer. Liu et al. (2019a) ap- ply hybrid semi-Markov conditional random field (HSCRF) to predict a set of candidate spans and rescore them with a pre-trained gazetteer model. In this paper, we combine …

Content truncated to 500 words