Learning to Generate Clinically Coherent Chest X-Ray Reports

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1235–1243 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 1235 Learning to Generate Clinically Coherent Chest X-Ray Reports Justin Lovelace Carnegie Mellon University jlovelac@andrew.cmu.edu Bobak Mortazavi Texas A&M University bobakm@tamu.edu Abstract Automated radiology report generation has the potential to reduce the time clinicians spend manually reviewing radiographs and stream- line clinical care. However, past work has shown that typical abstractive methods tend to produce fluent, but clinically incorrect radiol- ogy reports. In this work, we develop a radiol- ogy report generation model utilizing the trans- former architecture that produces superior re- ports as measured by both standard language generation and clinical coherence metrics com- pared to competitive baselines. We then de- velop a method to differentiably extract clini- cal information from generated reports and uti- lize this differentiability to fine-tune our model to produce more clinically coherent reports.1 1 Introduction Medical imaging (e.g. chest x-ray) is widely used in medicine for diagnostic purposes. However, cur- rent clinical practice requires a radiologist with specialized training to manually evaluate x-rays and note their findings in a radiology report. This manual evaluation is time-consuming and provid- ing an automated solution for this task would help streamline the clinical workflow and improve the quality of care. Image captioning has gained a large amount of attention following the curation of the COCO dataset (Lin et al., 2014) and the initial image cap- tioning work conducted on it (Vinyals et al., 2014; Xu et al., 2015; Lu et al., 2016a). Although im- age captioning is a widely studied task, radiology report generation offers unique challenges that pre- cludes the direct adaptation of many image cap- tioning models for the task. For example, much of the recent image captioning research has fol- lowed the work of Anderson et al. (2018) and uti- lized pre-trained objection detection models (Ren 1https://github.com/justinlovelace/ coherent-xray-report-generation et al., 2015) to extract image features (Yu et al., 2019; Huang et al., 2019; Yang et al., 2018; Yao et al., 2018). While this works well for general- domain datasets, analagous pre-trained models are not available in the clinical domain. Radiology reports are also typically longer and more complex than the captions available in stan- dard image captioning datasets such as COCO. Evaluation of medical report generation is also dif- ficult because the language generation metrics typ- ically used to evaluate image captioning systems can not directly evaluate the descriptive accuracy of generated reports which is of critical importance in the medical domain. There is already a body of past work that has focused on medical report generation, the most re- lated being that of Boag et al. (2019) and Liu et al. (2019) who both developed fully abstractive tech- niques for report generation. Boag et al. (2019) benchmarked a number of simple baselines on the MIMIC-CXR dataset (Johnson et al., 2019), the largest publicly available dataset of paired chest x- rays and radiology reports. They observed that typ- ical abstractive methods often produce …

Content truncated to 500 words