Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4038 - 4048 July 10-15, 2022 ©2022 Association for Computational Linguistics Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning Hongyi Yuan ∗ Zheng Yuan ∗ Sheng Yu † Tsinghua University {yuanhy20,yuanz17}@mails.tsinghua.edu.cn syu@tsinghua.edu.cn Abstract Entities lie in the heart of biomedical natu- ral language understanding, and the biomed- ical entity linking (EL) task remains chal- lenging due to the fine-grained and diversi- form concept names. Generative methods achieve remarkable performances in general domain EL with less memory usage while requiring expensive pre-training. Previous biomedical EL methods leverage synonyms from knowledge bases (KB) which is not triv- ial to inject into a generative method. In this work, we use a generative approach to model biomedical EL and propose to inject synonyms knowledge in it. We propose KB- guided pre-training by constructing synthetic samples with synonyms and definitions from KB and require the model to recover con- cept names. We also propose synonyms-aware fine-tuning to select concept names for train- ing, and propose decoder prompt and multi- synonyms constrained prefix tree for inference. Our method achieves state-of-the-art results on several biomedical EL tasks without can- didate selection which displays the effective- ness of proposed pre-training and fine-tuning strategies. The source code is available at Github.com/Yuanhy1997/GenBioEL. 1 Introduction Biomedical entity linking (EL) refers to mapping a biomedical mention (i.e., entity) in free texts to its concept in a biomedical knowledge base (KB) (e.g., UMLS (Bodenreider, 2004)). This is one of the most concerned tasks of research in medical natural language processing which is highly related to high-throughput phenotyping (Yu et al., 2015), relation extraction (Li et al., 2016), and automatic diagnosis (Yuan and Yu, 2021). Recent methods in biomedical EL mainly used neural networks to encode mentions and each con- cept name into the same dense space, then linked ∗ Contributed equally. † Corresponded author. mentions to corresponding concepts depending on embedding similarities (Sung et al., 2020; Lai et al., 2021; Bhowmik et al., 2021; Ujiie et al., 2021; Agarwal et al., 2021). Synonyms knowl- edge has been injected into these similarity-based methods by contrastive learning (Liu et al., 2021a; Yuan et al., 2022). For example in UMLS, con- cept C0085435 has synonyms: Reiter syndrome, Reactive arthritis and ReA which help models to learn different names of a concept entity. However, similarity-based methods requires large memory footprints to store representation for each concept which are hard to deploy. In general domain, GENRE (Cao et al., 2021a) viewed EL as a seq2seq task which inputs mentions with contexts and outputs concept names token- by-token. Mentions, contexts and concepts can be mingled due to the power of transformers, and the model does not need to store representations for each concept during inference. GENRE pre- trained on Wikipedia EL datasets to boost perfor- mances. However, directly implementing GENRE on biomedical EL cannot harvest satisfying re- sults. The gap occurs in two aspects: (1) There …

Content truncated to 500 words