Answering Clinical Questions with Knowledge-Based and Statistical Techniques

Answering Clinical Questions with Knowledge-Based and Statistical Techniques Dina Demner-Fushman∗ University of Maryland, College Park Jimmy Lin† University of Maryland, College Park The combination of recent developments in question-answering research and the availability of unparalleled resources developed specifically for automatic semantic processing of text in the medical domain provides a unique opportunity to explore complex question answering in the domain of clinical medicine. This article presents a system designed to satisfy the information needs of physicians practicing evidence-based medicine. We have developed a series of knowl- edge extractors, which employ a combination of knowledge-based and statistical techniques, for automatically identifying clinically relevant aspects of MEDLINE abstracts. These extracted elements serve as the input to an algorithm that scores the relevance of citations with respect to structured representations of information needs, in accordance with the principles of evidence- based medicine. Starting with an initial list of citations retrieved by PubMed, our system can bring relevant abstracts into higher ranking positions, and from these abstracts generate responses that directly answer physicians’ questions. We describe three separate evaluations: one focused on the accuracy of the knowledge extractors, one conceptualized as a document reranking task, and finally, an evaluation of answers by two physicians. Experiments on a collection of real-world clinical questions show that our approach significantly outperforms the already competitive PubMed baseline. 1. Introduction Recently, the focus of question-answering research has shifted away from simple fact- based questions that can be answered with relatively little linguistic knowledge to “harder” questions that require fine-grained text analysis, reasoning capabilities, and the ability to synthesize information from multiple sources. General purpose reasoning on anything other than superficial lexical relations is exceedingly difficult because there is a vast amount of world knowledge that must be encoded, either manually or auto- matically, to overcome the brittleness often associated with long chains of evidence. This situation poses a serious bottleneck to “advanced” question-answering systems. How- ever, the availability of existing knowledge sources and ontologies in certain domains provides exciting opportunities to experiment with knowledge-rich approaches. How might one go about leveraging these resources effectively? How might one integrate ∗ Department of Computer Science and Institute for Advanced Computer Studies. E-mail: demner@umd.edu. † College of Information Studies, Department of Computer Science, and Institute for Advanced Computer Studies. E-mail: jimmylin@umd.edu. Submission received: 4 July 2005; revised submission received: 7 January 2006; accepted for publication: 12 April 2006. © 2007 Association for Computational Linguistics Computational Linguistics Volume 33, Number 1 statistical techniques to overcome the brittleness often associated with knowledge- based approaches? We explore these interesting research questions in the domain of medicine, fo- cusing on the information needs of physicians in clinical settings. This domain is well-suited for exploring the posed research questions for several reasons. First, substantial understanding of the domain has already been codified in the Unified Medical Language System (UMLS) (Lindberg, Humphreys, and McCray 1993). Sec- ond, software for utilizing this ontology already exists: MetaMap (Aronson 2001) identifies concepts in free text, and SemRep (Rindflesch and Fiszman 2003) …

Content truncated to 500 words