Using Soft Constraints in Joint Inference for Clinical Concept Recognition

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1808–1814, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Using Soft Constraints in Joint Inference for Clinical Concept Recognition Prateek Jindal and Dan Roth Department of Computer Science, UIUC 201 N. Goodwin Ave, Urbana, IL 61801, USA {jindal2, danr}@illinois.edu Abstract This paper introduces IQPs (Integer Quadratic Programs) as a way to model joint inference for the task of concept recognition in clinical domain. IQPs make it possible to easily in- corporate soft constraints in the optimization framework and still support exact global infer- ence. We show that soft constraints give statis- tically significant performance improvements when compared to hard constraints. 1 Introduction In this paper, we study the problem of concept recognition in the clinical domain. State-of-the-art approaches (de Bruijn et al., 2011; Patrick et al., 2011; Torii et al., 2011; Minard et al., 2011; Jiang et al., 2011; Xu et al., 2012; Roberts and Harabagiu, 2011; Jindal and Roth, 2013) for concept recogni- tion in clinical domain (Uzuner et al., 2011) use sequence-prediction models like CRF (Lafferty et al., 2001), MEMM (McCallum et al., 2000) etc. These approaches are limited by the fact that they can model only local dependencies (most often, first-order models like linear chain CRFs are used to allow tractable inference). Clinical narratives, unlike newswire data, provide a domain with significant knowledge that can be ex- ploited systematically to improve the accuracy of the prediction task. Knowledge in this domain can be thought of as belonging to two categories: (1) Background Knowledge captured in medical ontolo- gies like UMLS (Url1, 2013), MeSH and SNOMED CT and (2) Discourse Knowledge driven by the fact that the narratives adhere to a specific writing style. While the former can be used by generating more expressive knowledge-rich features, the lat- ter is more interesting from our current perspective, since it provides global constraints on what output structures are likely and what are not. We exploit this structural knowledge in our global inference for- mulation. Integer Linear Programming (ILP) based ap- proaches have been used for global inference in many works (Roth and Yih, 2004; Punyakanok et al., 2004; Punyakanok et al., 2008; Marciniak and Strube, 2005; Bramsen et al., 2006; Barzilay and Lapata, 2006; Riedel and Clarke, 2006; Clarke and Lapata, 2007; Clarke and Lapata, 2008; Denis et al., 2007; Chang et al., 2011). However, in most of these works, researchers have focussed only on hard con- straints while formulating the inference problem. Formulating all the constraints as hard constraints is not always desirable because the constraints are not perfect in many cases. In this paper, we pro- pose Integer Quadratic Programs (IQPs) as a way of formulating the inference problem. IQPs is a richer family of models than ILPs and it enables us to easily incorporate soft constraints into the in- ference procedure. Our experimental results show that soft constraints indeed give much better perfor- mance than hard constraints. 2 Identifying Medical Concepts …

Content truncated to 500 words