That’s the Wrong Lung! Evaluating and Improving the Interpretability of Unsupervised Multimodal Encoders for Medical Data

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3626 - 3648 December 7-11, 2022 ©2022 Association for Computational Linguistics That’s the Wrong Lung! Evaluating and Improving the Interpretability of Unsupervised Multimodal Encoders for Medical Data Denis Jered McInerney Northeastern University mcinerney.de@northeastern.edu Geoffrey Young Brigham and Women’s Hospital gsyoung@bwh.harvard.edu Jan-Willem van de Meent University of Amsterdam j.w.vandemeent@uva.nl Byron C. Wallace Northeastern University b.wallace@northeastern.edu Abstract Pretraining multimodal models on Electronic Health Records (EHRs) provides a means of learning representations that can transfer to downstream tasks with minimal supervision. Recent multimodal models induce soft local alignments between image regions and sen- tences. This is of particular interest in the medical domain, where alignments might high- light regions in an image relevant to specific phenomena described in free-text. While past work has suggested that attention “heatmaps” can be interpreted in this manner, there has been little evaluation of such alignments. We compare alignments from a state-of-the-art mul- timodal (image and text) model for EHR with human annotations that link image regions to sentences. Our main finding is that the text has an often weak or unintuitive influence on attention; alignments do not consistently re- flect basic anatomical information. Moreover, synthetic modifications — such as substituting “left” for “right” — do not substantially influ- ence highlights. Simple techniques such as al- lowing the model to opt out of attending to the image and few-shot finetuning show promise in terms of their ability to improve alignments with very little or no supervision. We make our code and checkpoints open-source.1 1 Introduction There has been a flurry of recent work on model ar- chitectures and self-supervised training objectives for multimodal representation learning, both gener- ally (Li et al., 2019; Tan and Bansal, 2019; Huang et al., 2020; Su et al., 2020; Chen et al., 2020) and for medical data specifically (Wang et al., 2018; Chauhan et al., 2020; Li et al., 2020). These meth- ods yield representations that permit efficient learn- ing on various multimodal downstream tasks (e.g., classification, captioning). Given the inherently multimodal nature of much medical data — e.g., in radiology images and text 1https://github.com/dmcinerney/gloria are naturally paired — there has been particular interest in designing multimodal models for Elec- tronic Health Records (EHRs) data. However, one of the factors that currently stands in the way of broader adoption is interpretability. Neural models that map image-text pairs to shared representations are opaque. Consequently, doctors have no way of knowing whether such models rely on meaningful clinical signals or data artifacts (Zech et al., 2018). Recent work has proposed models that soft-align text snippets to image regions. This may afford a type of interpretability by allowing practition- ers to inspect what the model has “learned” or al- low more efficient identification of relevant regions. Past work has presented illustrative multimodal “saliency” maps in which such models highlight plausible regions. But such highlights also risk pro- viding a false sense that the model “understands” more than it actually does, and irrelevant highlights …

Content truncated to 500 words