Benchmarking Automated Clinical Language Simplification: Dataset, Algorithm, and Evaluation

Proceedings of the 29th International Conference on Computational Linguistics, pages 3550–3562 October 12–17, 2022. 3550 Benchmarking Automated Clinical Language Simplification: Dataset, Algorithm, and Evaluation Junyu Luo1, Junxian Lin2,3, Chi Lin2,3, Cao Xiao4, Xinning Gui1, Fenglong Ma1∗ 1College of Information Sciences and Technology, Pennsylvania State University, USA 2School of Software Technology, Dalian University of Technology, China 3Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, China 4Relativity, USA {junyu, xinninggui, fenglong}@psu.edu, linjunxian@mail.dlut.edu.cn, c.lin@dlut.edu.cn, cao.xiao@relativity.com Abstract Patients with low health literacy usually have difficulty understanding medical jargon and the complex structure of professional medical language. Although some studies are proposed to automatically translate expert language into layperson-understandable language, only a few of them focus on both accuracy and read- ability aspects simultaneously in the clinical domain. Thus, simplification of the clinical language is still a challenging task, but unfor- tunately, it is not yet fully addressed in previ- ous work. To benchmark this task, we con- struct a new dataset named MedLane to sup- port the development and evaluation of au- tomated clinical language simplification ap- proaches. Besides, we propose a new model called DECLARE that follows the human anno- tation procedure and achieves state-of-the-art performance compared with eight strong base- lines. To fairly evaluate the performance, we also propose three specific evaluation metrics. Experimental results demonstrate the utility of the annotated MedLane dataset and the effec- tiveness of the proposed model DECLARE1 1 Introduction Health literacy is generally defined as the ability of patients to obtain, process, understand, and com- municate basic health information (Parker et al., 1999), which is significantly important for mak- ing health decisions and ensuring treatment out- comes. The increasing accessibility of technology information makes patients have more opportuni- ties to access health information. However, it is challenging for patients, especially with limited health literacy, to read and understand health mate- rials such as clinical notes written by doctors, with medical jargon (Korsch et al., 1968), abbreviations, and professional language (Friedman et al., 2002). ∗Corresponding author. 1The source code of DECLARE and the MedLane dataset can be found in the https://github.com/ machinelearning4health/MedLane. The lack of proper communication between doc- tors and patients not only results in a tense doctor- patient relationship (Ha and Longnecker, 2010) but also increases the risk of adverse health outcomes over time (Sudore et al., 2006). Therefore, there is a great need to simplify professional clinical lan- guage to layperson-understandable language. 1.1 Why We Need a New Dataset? To implement an automated clinical language sim- plification system, the first step is to prepare the dataset for model training. Although there are sev- eral annotated medical datasets that are summa- rized in Table 1, most of them do not focus on the clinical domain. As we discussed before, the writing style of clinical materials is significantly different from that used in the general biomedical area. Thus, those datasets cannot be used to train a clinical simplification model. To the best of our knowledge, there are only two datasets focusing on clinical language …

Content truncated to 500 words