Reinforcement Learning with Imbalanced Dataset for Data-to-Text Medical Report Generation

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2223–2236 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 2223 Reinforcement Learning with Imbalanced Dataset for Data-to-Text Medical Report Generation Toru Nishino1 Ryuji Kano1 Ryota Ozaki1 Norihisa Nakano1 Tomoko Ohkuma1 1Fuji Xerox Co., Ltd. Yohei Momoki2 Yuki Tagawa1 Keigo Nakamura2 2Fujifilm Corporation Tomoki Taniguchi1 Motoki Taniguchi1 nishino.toru@fujixerox.co.jp, toru.nishino@fujifilm.com Abstract Automated generation of medical reports that describe the findings in the medical images helps radiologists by alleviating their work- load. Medical report generation system should generate correct and concise reports. However, data imbalance makes it difficult to train mod- els accurately. Medical datasets are commonly imbalanced in their finding labels because in- cidence rates differ among diseases; moreover, the ratios of abnormalities to normalities are significantly imbalanced. We propose a novel reinforcement learning method with a recon- structor to improve the clinical correctness of generated reports to train the data-to-text mod- ule with a highly imbalanced dataset. More- over, we introduce a novel data augmentation strategy for reinforcement learning to addition- ally train the model on infrequent findings. From the perspective of a practical use, we employ a Two-Stage Medical Report Genera- tor (TS-MRGen) for controllable report gener- ation from input images. TS-MRGen consists of two separated stages: an image diagnosis module and a data-to-text module. Radiolo- gists can modify the image diagnosis module results to control the reports that the data-to- text module generates. We conduct an experi- ment with two medical datasets to assess the data-to-text module and the entire two-stage model. Results demonstrate that the reports generated by our model describe the findings in the input image more correctly. 1 Introduction Writing medical reports manually from medical im- ages is a time-consuming task for radiologists. To write reports, radiologists first recognize what find- ings are included in medical images, such as com- puted tomography (CT) and X-ray images. Then radiologists compose reports that describe the rec- ognized findings correctly without omission. Doc- tors prefer radiology reports written in natural lan- guage. Other types of radiology reports, such as Figure 1: Overview of our Two-Stage Medical Report Generator (TS-MRGen) system. Figure 2: Imbalanced distribution of the part of the find- ing labels in the MIMIC-CXR dataset. tabular reports, are difficult to understand because of their complexity. The purpose of our work is to build an auto- mated medical report generation system to reduce the workload of radiologists. As shown in Fig- ure 1, the medical report generation system should generate correct and concise reports for the input images. However, data imbalance may reduce the quality of automatically generated reports. Med- ical datasets are commonly imbalanced in their finding labels because incidence rates differ among 2224 diseases; moreover, the ratios of abnormalities to normalities are also significantly imbalanced. Fig- ure 2 shows an imbalanced distribution of find- ing labels in the MIMIC-CXR dataset (Johnson et al., 2019). For example, the finding label “En- larged Cardiomediastinum.Negative” appears ap- proximately 70 times more frequently than the find- ing label “Atelectasis.Negative”. …

Content truncated to 500 words