DeltaNet: Conditional Medical Report Generation for COVID-19 Diagnosis

Proceedings of the 29th International Conference on Computational Linguistics, pages 2952–2961 October 12–17, 2022. 2952 DeltaNet: Conditional Medical Report Generation for COVID-19 Diagnosis Xian Wu1, Shuxin Yang2,5, Zhaopeng Qiu1, Shen Ge1, Yangtian Yan1, Xingwang Wu3 Yefeng Zheng1, S. Kevin Zhou4, Li Xiao2,5∗ 1Tencent Jarvis Lab 2Institute of Computing Technology of the Chinese Academy of Sciences 3The First Affiliated Hospital of Anhui Medical University 4University of Science and Technology of China 5University of Chinese Academy of Sciences {kevinxwu, shuxinyang, zhaopengqiu, shenge, yangtianyan, yefengzheng}@tencent.com; duobi2004@126.com; s.kevin.zhou@gmail.com;xiaoli@ict.ac.cn Abstract Fast screening and diagnosis are critical in COVID-19 patient treatment. In addition to the gold standard RT-PCR, radiological imaging like X-ray and CT also works as an important means in patient screening and follow-up. How- ever, due to the excessive number of patients, writing reports becomes a heavy burden for radiologists. To reduce the workload of radiolo- gists, we propose DeltaNet to generate medical reports automatically. Different from typical image captioning approaches that generate re- ports with an encoder and a decoder, DeltaNet applies a conditional generation process. In particular, given a medical image, DeltaNet employs three steps to generate a report: 1) first retrieving related medical reports, i.e., the historical reports from the same or similar pa- tients; 2) then comparing retrieved images and current image to find the differences; 3) finally generating a new report to accommodate identi- fied differences based on the conditional report. We evaluate DeltaNet on a COVID-19 dataset, where DeltaNet outperforms state-of-the-art ap- proaches. Besides COVID-19, the proposed DeltaNet can be applied to other diseases as well. We validate its generalization capabili- ties on the public IU-Xray and MIMIC-CXR datasets for chest-related diseases. 1 Introduction Since December 2019, the world has been suffer- ing from a serious health crisis: the outbreak of COVID-19 (Wang et al., 2020). Fast screening and diagnosis is critical in COVID-19 patient treatment. In clinical practice, the Reverse Transcription Poly- merase Chain Reaction (RT-PCR) is recognized as the golden standard (Zu et al., 2020). How- ever, due to high false-negative rate and shortage of equipment (Fang et al., 2020; Ng et al., 2020), medical imaging like X-ray and Computed Tomog- raphy (CT) (Rubin et al., 2020) also works as ∗Corresponding author an alternative means in COVID-19 diagnosis and treatment which generates more timely results than RT-PCR and helps in evaluating the severity degree of COVID-19. Given medical images of COVID-19 patients, radiologists need to write relatively long reports to address the impressions and findings. Consid- ering the large volume of COVID-19 patients and potentially infected population, writing medical reports becomes a heavy burden for radiologists. Furthermore, due to the varied expertise of radiolo- gists, some abnormalities in medical images may be ignored and thus not included in the final re- ports. To alleviate the heavy workload and aid less experienced radiologists, automatically generating medical reports becomes a critical task. Due to its importance in clinical practice, auto- matically generating medical reports has attracted extensive research interests in recent years (Zhou et al., 2021). Existing works …

Content truncated to 500 words