Factual Accuracy is not Enough: Planning Consistent Description Order for Radiology Report Generation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7123 - 7138 December 7-11, 2022 ©2022 Association for Computational Linguistics Factual Accuracy is not Enough: Planning Consistent Description Order for Radiology Report Generation Toru Nishino1 Yasuhide Miura1 Tomoki Taniguchi1 Tomoko Ohkuma1 Yuki Suzuki2 Shoji Kido2 Noriyuki Tomiyama2 1Fujifilm Corporation 2Osaka University Graduate School of Medicine toru.nishino@fujifilm.com Abstract Radiology report generation systems have the potential to reduce the workload of radiolo- gists by automatically describing the findings in medical images. To broaden the applica- tion of the report generation system, the sys- tem should generate reports that are not only factually accurate but also chronologically con- sistent, describing images that are presented in time order, that is, the correct order. We employ a planning-based radiology report generation system that generates the overall structure of re- ports as “plans” prior to generating reports that are accurate and consistent in order. Addition- ally, we propose a novel reinforcement learning and inference method, Coordinated Planning (CoPlan), that includes a content planner and a text generator to train and infer in a coordinated manner to alleviate the cascading of errors that are often inherent in planning-based models. We conducted experiments with single-phase diagnostic reports in which the factual accuracy is critical and multi-phase diagnostic reports in which the description order is critical. Our pro- posed CoPlan improves the content order score by 5.1 pt in time series critical scenarios and the clinical factual accuracy F-score by 9.1 pt in time series irrelevant scenarios, compared those of the baseline models without CoPlan. 1 Introduction Radiologists regularly write qualitative radiology reports to accurately describe the recognized find- ings in medical images. Recently, we can observe two different approaches to radiology imaging: a single-phase and a multiphase imaging method. A single-phase diagnostic approach, as applied in a plain X-ray machine, scans only once, while many modern procedures, including liver contrast CT, use a multiphase diagnostic method, scanning se- quentially in a period of several minutes. The time- dependent scans are labeled as phases. Reports of single-phase diagnoses are time series irrelevant, so the factual accuracy is the most critical quality !"#$%&'()*&+" !"#$%&'()*+,'-./$01 ,"-./0 !"#$%&'()*&+"(1$02(3$*"(4"/$"5 !2%3*/'(4&,/$+,'(51 ,"-./0 !"#$"%&' !"#$% ()"#&'*+$,)-.*/0&.$ 1$'&2$3*(0&.$ 4%5$*.$"%$. !"#$"#%&'$()$*(+,+-$#.+%&/#, %.,$ '"#&#$()$%$0#&)()'#.'$(.1&#%)#,$ +#2'$&#'&31%&,(%1$30%1('-4$ !"#&#$()$.3$0+#5&%+$#225)(3. 3&$ 0.#5*3'"3&%64 7.$'"#$#&'%&(#)*!"#$%8$'"#$+#)(3.$ )"39)$%.$#%&+-$%&'#&(%+$ #."%.1#*#.'4$:3*(''#,;$<(./$ #."%.1#*#.'$()$)"39#,$,5&(./$'"#$ +%)#,%+*!"#$%*13.)()'#.'$9('"$=>>4 Figure 1: Comparison of reports of single-phase diag- nosis and multiphase diagnosis with time series. !"#$"#%&'$()$*(+,+-$#.+%&/#, %.,$'"#&#$()$%$0#&)()'#.'$ (.1&#%)#,$+#2'$&#'&31%&,(%1$30%1('-4$ !"#&#$()$.3$0+#5&%+$#225)(3. 3&$0.#5*3'"3&%64 !"#$%&'()*&+" !"#$% &'#(()*)%+ !"#$% 78%&,(3*#/%+-9$:; 7<5./$=0%1('-9$:; 7:+#5&%+$>225)(3.9$?; 7:.#5*3'"3&%69$?; ! !"#$%#$&'()##%*&+!', ,-"#$%."#(,'&/ !"#"$#"%&'()%()*+ -%.$&/%#%*)$"*&+-/, 0"/"-&."#(1"23-. ,#-#.-/.0%1-23)-425'#66)6$27,809 @4$ 78%&,(3*#/%+-9$:; 7<5./$=0%1('-9$:; A4$ 7:+#5&%+B>225)(3.9$?; 7:.#5*3'"3&%69$?; !!! Figure 2: Overview of our planning-based report gener- ation system. consideration. However, reports of multiphase di- agnoses are time series critical, so a time-variable transition of lesions in the image must be correctly described with consistent description order. For example ,in Figure 1, a liver contrast CT report is composed to reflect the order of time series, with an arterial phase followed by a delayed phase; on the contrary, the chest X-ray reports are composed with no regard to …

Content truncated to 500 words