Augmenting word2vec with latent Dirichlet allocation within a clinical application

Proceedings of NAACL-HLT 2019, pages 4095–4099 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 4095 Augmenting word2vec with Latent Dirichlet Allocation within a Clinical Application Akshay Budhkar University of Toronto; Vector Institute for Artificial Intelligence abudhkar@cs.toronto.edu Frank Rudzicz University of Toronto; St Michael’s Hospital; Surgical Safety Technologies Inc.; Vector Institute for Artificial Intelligence frank@cs.toronto.edu Abstract This paper presents three hybrid models that directly combine latent Dirichlet allocation and word embedding for distinguishing be- tween speakers with and without Alzheimer’s disease from transcripts of picture descrip- tions. Two of our models get F-scores over the current state-of-the-art using automatic meth- ods on the DementiaBank dataset. 1 Introduction Word embedding projects words into a lower- dimensional latent space that captures semantic and morphological information. Separately but re- lated, the task of topic modelling also discovers la- tent semantic structures or topics in a corpus. La- tent Dirichlet allocation (LDA) uses bag-of-words statistics to infer topics in an unsupervised man- ner. LDA considers each document to be a prob- ability distribution over hidden topics, and each topic is a probability distribution over all words in the vocabulary, both with Dirichlet priors. The inferred probabilities over learned latent topics of a given document (i.e., topic vectors) can be used along with a discriminative classifier, as in the work by Luo and Li (2014), but other ap- proaches such as TF-IDF (Lan et al., 2005) eas- ily outperform this model, like in the case of the Reuters-21578 corpus (Lewis et al., 1987). Here, we hypothesize that creating a hybrid of LDA and word2vec (Mikolov et al., 2013b) models will pro- duce discriminative features. We introduce three new variants of hybrid LDA-word2vec models, and investigate the effect of dropping the first com- ponent after principal component analysis (PCA). These models can be thought of as extending the conglomeration of topical embedding models. We incorporate topical information into our word2vec models by using the final state of the topic-word distribution in the LDA model during training. 1.1 Motivation and Related Work Alzheimer’s disease (AD) is a neurodegenera- tive disease that affects approximately 5.5 million Americans with annual costs of care up to $259B in the US, in 2017, alone (Alzheimer’s Association et al., 2017). The existing state-of-the-art methods for detecting AD from speech used extensive fea- ture engineering, some of which involved experi- enced clinicians. Fraser et al. (2016) investigated multiple linguistic and acoustic characteristics and obtained accuracies up to 81% with aggressive feature selection. Yancheva and Rudzicz (2016) use vector-space topic models, and achieved F- scores up to 74%. It is generally expensive to get sufficient labeled data for arbitrary patholog- ical conditions. In our experiments, we train our hybrid models on a normative dataset and apply them for classifi- cation on a clinical dataset. The goal of this project is to i) effectively augment word2vec with LDA for classification, and ii) to improve the accuracy of dementia detection using automatic methods. 2 Datasets The Wisconsin Longitudinal Study (WLS) is a …

Content truncated to 500 words