Medical Concept Embeddings via Labeled Background Corpora

Medical Concept Embeddings via Labeled Background Corpora Eneldo Loza Menc´ıa1, Gerard de Melo2, Jinseok Nam3 1 Knowledge Engineering Group, Technische Universit¨at Darmstadt, Germany 2 IIIS, Tsinghua University, Beijing, China 3 Knowledge Discovery in Scientific Literature, Technische Universit¨at Darmstadt, Germany eneldo@ke.tu-darmstadt.de, gdm@demelo.org, nam@kdsl.informatik.tu-darmstadt.de Abstract In recent years, we have seen an increasing amount of interest in low-dimensional vector representations of words. Among other things, these facilitate computing word similarity and relatedness scores. The most well-known example of algorithms to produce representations of this sort are the word2vec approaches. In this paper, we investigate a new model to induce such vector spaces for medical concepts, based on a joint objective that exploits not only word co-occurrences but also manually labeled documents, as available from sources such as PubMed. Our extensive experimental analysis shows that our embeddings lead to significantly higher correlations with human similarity and relatedness assessments than previous work. Due to the simplicity and versatility of vector representations, these findings suggest that our resource can easily be used as a drop-in replacement to improve any systems relying on medical concept similarity measures. Keywords: medical concepts, semantic similarity, MeSH, embeddings 1 Introduction For many decades, researchers have debated the curious nature of human language. On the one hand, language ap- pears to naturally involve discrete symbolic units. On the other hand, these symbolic units are clearly not indepen- dent. There is a clear relationship, for instance, between the two words neuron and neural. The same applies to multi- word expressions, as are common in the medical domain, for instance, between the central nervous system and the peripheral nervous system. In the past, many models ignored such semantic relation- ships, or invoked custom techniques such as query expan- sion, to cope with them. Recently, another alternative has proven useful and acquired significant popularity: Discrete symbols can be mapped to a low-dimensional vector space, in which distances or angles between vectors reflect simi- larities between the symbols. Many of the approaches for this rely on neural network techniques. However, existing approaches such as the word2vec Skip-Gram with Negative Sampling model (Mikolov et al., 2013) neglect valuable additional information that may be available in the used doc- ument collections, e.g. human annotations or hierarchical information. In this paper, we investigate a new approach called All-in- text for producing embeddings for medical concepts (Nam et al., 2016). All-in-text was originally proposed for key- word indexing and multi-label classification. In this paper, we adapt it to the task of producing embeddings for medical concepts. The learning method is inspired by the Paragraph Vector technique (Le and Mikolov, 2014) of learning repre- sentations of words and word sequences (documents), which was originally used for text classification and sentiment anal- ysis. However, All-in-text is a versatile framework that can incorporate more complex connectivity information origi- nating from the associations between documents and target concepts in a background corpus. Fig. 1 shows some ex- ample documents from the used corpus, associated with the concepts Malnutrition and Overnutrition. Our experimental evaluation …

Content truncated to 500 words