Drug Extraction from the Web: Summarizing Drug Experiences with Multi-Dimensional Topic Models

Proceedings of NAACL-HLT 2013, pages 168–178, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Drug Extraction from the Web: Summarizing Drug Experiences with Multi-Dimensional Topic Models Michael J. Paul and Mark Dredze Human Language Technology Center of Excellence Center for Language and Speech Processing Johns Hopkins University Baltimore, MD 21218 {mpaul,mdredze}@cs.jhu.edu Abstract Multi-dimensional latent text models, such as factorial LDA (f-LDA), capture multiple fac- tors of corpora, creating structured output for researchers to better understand the contents of a corpus. We consider such models for clinical research of new recreational drugs and trends, an important application for mining current information for healthcare workers. We use a “three-dimensional” f-LDA variant to jointly model combinations of drug (mari- juana, salvia, etc.), aspect (effects, chemistry, etc.) and route of administration (smoking, oral, etc.) Since a purely unsupervised topic model is unlikely to discover these specific factors of interest, we develop a novel method of incorporating prior knowledge by leverag- ing user generated tags as priors in our model. We demonstrate that this model can be used as an exploratory tool for learning about these drugs from the Web by applying it to the task of extractive summarization. In addition to providing useful output for this important pub- lic health task, our prior-enriched model pro- vides a framework for the application of f- LDA to other tasks. 1 Introduction Topic models aid exploration of the main thematic elements of large text corpora by revealing latent structure and producing a high level semantic view (Blei et al., 2003). Topic models have been used for understanding the contents of a corpus and identify- ing interesting aspects of a collection for more in- depth analysis (Talley et al., 2011; Mimno, 2011). While standard topic models assume a flat seman- tic structure, there are potentially many dimen- sions of a corpus that contribute to word choice, such as sentiment, perspective and ideology (Mei et al., 2007; Paul and Girju, 2010; Eisenstein et al., 2011). Rather than studying these factors in isola- tion, multi-dimensional topic models can consider multiple factors jointly. Paul and Dredze (2012b) introduced factorial LDA (f-LDA), a general framework for multi- dimensional text models that capture an arbitrary number of factors (explained in §3). While a stan- dard topic model learns distributions over “topics” in documents, f-LDA learns distributions over com- binations of multiple factors (e.g. topic, perspec- tive) called tuples (e.g. (HEALTHCARE,LIBERAL)). While f-LDA can model factors without supervision, it has not been used in situations where the user has prior information about the factors. In this paper we consider a setting where the user has prior knowledge about the end application: min- ing recreational drug trends from user forums, an important clinical research problem (§2). We show how to incorporate available information from these forums into f-LDA as a novel hierarchical prior over the model parameters, guiding the model toward the desired output (§3.1). We then demonstrate the model’s utility in ex- ploring a corpus in a targeted manner by using it …

Content truncated to 500 words