A Generate-and-Rank Framework with Semantic Type Regularization for Biomedical Concept Normalization

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8452–8464 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 8452 A Generate-and-Rank Framework with Semantic Type Regularization for Biomedical Concept Normalization Dongfang Xu and Zeyu Zhang and Steven Bethard School of Information University of Arizona Tucson, AZ {dongfangxu9,zeyuzhang,bethard}@email.arizona.edu Abstract Concept normalization, the task of linking tex- tual mentions of concepts to concepts in an on- tology, is challenging because ontologies are large. In most cases, annotated datasets cover only a small sample of the concepts, yet con- cept normalizers are expected to predict all concepts in the ontology. In this paper, we propose an architecture consisting of a candi- date generator and a list-wise ranker based on BERT. The ranker considers pairings of con- cept mentions and candidate concepts, allow- ing it to make predictions for any concept, not just those seen during training. We further en- hance this list-wise approach with a semantic type regularizer that allows the model to in- corporate semantic type information from the ontology during training. Our proposed con- cept normalization framework achieves state- of-the-art performance on multiple datasets. 1 Introduction Mining and analyzing the constantly-growing un- structured text in the bio-medical domain offers great opportunities to advance scientific discovery (Gonzalez et al., 2015; Fleuren and Alkema, 2015) and improve the clinical care (Rumshisky et al., 2016; Liu et al., 2019). However, lexical and gram- matical variations are pervasive in such text, posing key challenges for data interoperability and the de- velopment of natural language processing (NLP) techniques. For instance, heart attack, MI, myocar- dial infarction, and cardiovascular stroke all refer to the same concept. It is critical to disambiguate these terms by linking them with their correspond- ing concepts in an ontology or knowledge base. Such linking allows downstream tasks (relation ex- traction, information retrieval, text classification, etc.) to access the ontology’s rich knowledge about biomedical entities, their synonyms, semantic types and mutual relationships. Concept normalization is a task that maps con- cept mentions, the in-text natural-language men- tions of ontological concepts, to concept entries in a standardized ontology or knowledge base. Tech- niques for concept normalization have been ad- vancing, thanks in part to recent shared tasks in- cluding clinical disorder normalization in 2013 ShARe/CLEF (Suominen et al., 2013) and 2014 SemEval Task 7 Analysis of Clinical Text (Pradhan et al., 2014), and adverse drug event normaliza- tion in Social Media Mining for Health (SMM4H) (Sarker et al., 2018; Weissenbacher et al., 2019). Most existing systems use a string-matching or dictionary look-up approach (Leal et al., 2015; D’Souza and Ng, 2015; Lee et al., 2016), which are limited to matching morphologically similar terms, or supervised multi-class classifiers (Be- lousov et al., 2017; Tutubalina et al., 2018; Niu et al., 2019; Luo et al., 2019a), which may not generalize well when there are many concepts in the ontology and the concept types that must be predicted do not all appear in the training data. We propose an architecture (shown …

Content truncated to 500 words