Analysis of Word Embeddings and Sequence Features for Clinical Information Extraction

Analysis of Word Embeddings and Sequence Features for Clinical Information Extraction Lance De Vine, Mahnoosh Kholghi, Guido Zuccon, Laurianne Sitbon Queensland University of Technology {l.devine, m1.kholghi, g.zuccon, laurianne.sitbon}@qut.edu.au Anthony Nguyen The Australian e-Health Research Centre, CSIRO anthony.nguyen@csiro.au Abstract This study investigates the use of unsuper- vised features derived from word embed- ding approaches and novel sequence rep- resentation approaches for improving clin- ical information extraction systems. Our results corroborate previous findings that indicate that the use of word embeddings significantly improve the effectiveness of concept extraction models; however, we further determine the influence that the corpora used to generate such features have. We also demonstrate the promise of sequence-based unsupervised features for further improving concept extraction. 1 Introduction Clinical concept extraction involves the identifica- tion of sequences of terms which express mean- ingful concepts in a clinical setting. The identifi- cation of such concepts is important for enabling secondary usage of reports of patient treatments and interventions, e.g., in the context of cancer monitoring and reporting (Koopman et al., 2015), and for further processing in downstream eHealth workflows (Demner-Fushman et al., 2009). A significant challenge is the identification of concepts that are referred to in ways not captured within current lexical resources such as relevant domain terminologies like SNOMED CT. Further- more, clinical language is sensitive to ambiguity, polysemy, synonymy (including acronyms) and word order variations. Finally, the information presented in clinical narratives is often unstruc- tured, ungrammatical, and fragmented. State of the art approaches in concept extraction from free-text clinical narratives extensively apply supervised machine learning approaches. The ef- fectiveness of such approaches generally depends on three main factors: (1) the availability of a con- siderable amount of high quality annotated data, (2) the selected learning algorithm, and (3) the quality of features generated from the data. In recent years, clinical information extraction and retrieval challenges like i2b2 (Uzuner et al., 2011) and ShARe/CLEF (Suominen et al., 2013) have provided annotated data which can be used to apply and evaluate different machine learning ap- proaches (e.g., supervised and semi-supervised). Conditional Random Fields (CRFs) (Lafferty et al., 2001) has shown to be the state-of-the-art su- pervised machine learning approach for this clini- cal task. A wide range of features has been lever- aged to improve the effectiveness of concept ex- traction systems, including hand-crafted grammat- ical, syntactic, lexical, morphological and ortho- graphical features (de Bruijn et al., 2011; Tang et al., 2013), as well as advanced semantic fea- tures from external resources and domain knowl- edge (Kholghi et al., 2015). While there has been some recent work in the application of unsupervised machine learning methods to clinical concept extraction (Jonnala- gadda et al., 2012; Tang et al., 2013), the predom- inant class of features that are used are still hand- crafted features. This paper discusses the application to clini- cal concept extraction of a specific unsupervised machine learning method, called the Skip-gram Neural Language Model, combined with a lexi- cal string encoding approach and sequence fea- tures. Skip-gram word embeddings, …

Content truncated to 500 words