Conceptual Grounding Constraints for Truly Robust Biomedical Name Representations

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2440–2450 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2440 Conceptual Grounding Constraints for Truly Robust Biomedical Name Representations Pieter Fivez CLiPS Research Centre University of Antwerp pieter.fivez@uantwerpen.be Simon ˇSuster Faculty of Engineering and Information Technology University of Melbourne simon.suster@unimelb.edu.au Walter Daelemans CLiPS Research Centre University of Antwerp walter.daelemans@uantwerpen.be Abstract Effective representation of biomedical names for downstream NLP tasks requires the encod- ing of both lexical as well as domain-specific semantic information. Ideally, the synonymy and semantic relatedness of names should be consistently reflected by their closeness in an embedding space. To achieve such robustness, prior research has considered multi-task objec- tives when training neural encoders. In this pa- per, we take a next step towards truly robust representations, which capture more domain- specific semantics while remaining universally applicable across different biomedical corpora and domains. To this end, we use conceptual grounding constraints which more effectively align encoded names to pretrained embed- dings of their concept identifiers. These con- straints are effective even when using a Deep Averaging Network, a simple feedforward en- coding architecture that allows for scaling to large corpora while remaining sufficiently ex- pressive. We empirically validate our ap- proach using multiple tasks and benchmarks, which assess both literal synonymy as well as more general semantic relatedness. Our code is open-source and available at www.github. com/clips/conceptualgrounding. 1 Introduction Biomedical and clinical free-text contain mentions of biomedical terms which can provide valuable information for text mining applications. Such tex- tual mentions, as well as their corresponding ref- erence names in biomedical ontologies, can often be expressed in various synonymous surface forms (e.g. pleuritic pain vs. pain breathing), which is challenging for downstream applications. Effective dense representation of these biomedical names ICD-10 SNOMED-CT C0564504 schizoid fantasy schizoid fantasy - mental defense mechanism F60.1 C0338969 introverted personality disorder introverted personality C0036339 schizoid personality disorder unspecified schizoid personality disorder Table 1: Example of SNOMED-to-ICD-10 mappings. The synonym sets for the SNOMED-CT concepts C0564504, C0338969, and C0036339, are fused into one large set of semantically related names for the ICD- 10 code F60.1. has been mainly investigated through the normal- ization task of disorder linking, which consists of matching disease mentions to reference terms of concept identifiers in ontologies (e.g. matching the mention myocardial depression to the refer- ence term Myocardial Dysfunction) (Leaman et al., 2015). While past research has gradually shifted its focus from lexical representations (Leaman et al., 2013; D’Souza and Ng, 2015) to dense distributed representations (Limsopatham and Collier, 2016; Li et al., 2017; Phan et al., 2019; Sung et al., 2020), encoders are still typically optimized towards nor- malization tasks, which are focused on resolving word-level analogies between synonymous biomed- ical names. Recent research has focused more explicitly on encoding domain-specific biomedical semantics by training biomedical name representations that are robust, i.e., reflecting the synonymy and seman- tic relatedness of names by their closeness in the embedding space, preferably in …

Content truncated to 500 words