Towards multimodal modeling of physicians’ diagnostic confidence and self-awareness using medical narratives

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1718–1727, Dublin, Ireland, August 23-29 2014. Towards multimodal modeling of physicians’ diagnostic confidence and self-awareness using medical narratives Joseph Bullard† Cecilia Ovesdotter Alm‡ Qi Yu† Pengcheng Shi† Anne Haake† †College of Computing and Information Sciences ‡College of Liberal Arts Rochester Institute of Technology jtb4478@cs.rit.edu coagla|qi.yu|spcast|arhics@rit.edu Abstract Misdiagnosis is a problem in the medical field, often related to physicians’ cognitive errors. Overconfidence is considered a major cause of such errors. Intelligent diagnostic support sys- tems could benefit from understanding how aware physicians are of their performance when they estimate their confidence in a diagnosis (i.e. a physician’s diagnostic self-awareness). Shed- ding light on the cognitive processes related to such awareness could also help improve medical education. We use a multimodal dataset of medical narratives to computationally model diagnos- tic confidence and self-awareness based on physicians’ linguistic and eye movement behaviors. Dermatologists viewed images of cutaneous conditions, providing a description, diagnosis, and certainty level for each image case, while their speech and eye movements were recorded. We define both a generalized and a personalized approach to binning confidence levels, used in clas- sification experiments. We also introduce truly multimodal features, which focus on combining linguistic and eye movement data into multimodal attributes. Results indicate that combinations of multiple modalities can outperform their constituent modalities in isolation for these problems. 1 Introduction Misdiagnosis in the medical field is estimated to be as high as 10%-15% (Berner and Graber, 2008; Croskerry, 2009). Such errors can result in incorrect or delayed treatment, causing patients to experience additional suffering. Graber et al. (2002) describe three types of diagnostic errors: no-fault errors, result- ing from atypical disease presentation or limitations of medical knowledge; system errors, resulting from problems with the health care system; and cognitive errors, resulting from biases or faulty interpretation on the part of a physician. Cognitive errors in particular have potential for substantial reduction through education and training aimed at developing clinicians’ metacognitive skills. Understanding the cognitive processes of physicians during diagnosis is also of critical importance for building human-centered di- agnostic support systems, which could help detect and flag problematic diagnostic self-awareness cases. Examples of cognitive errors include settling on a final diagnosis too early, without ever considering the correct diagnosis (Berner and Graber, 2008), or confirmation bias, in which only evidence to confirm a diagnostic hypothesis is considered (Croskerry, 2003). Overconfidence is generally thought to be a major cause of such errors (Berner and Graber, 2008; Croskerry, 2008). For example, an overconfident physi- cian may not question her original thoughts or explore alternative diagnoses until later in the treatment process. In general, overconfidence may be a systemic problem, reinforced by patients’ preferences for confident doctors, and by a professional environment that favors decisive actions (Katz, 1984). Similarly, underconfidence can erode patients’ trust in their providers. In this study, we view the interplay between confidence1 and correctness as a two-dimensional problem (see Figure 1). Ideally, physicians would have high …

Content truncated to 500 words