Document-Level Text Simplification: Dataset, Criteria and Baseline

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7997–8013 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 7997 Document-Level Text Simplification: Dataset, Criteria and Baseline Renliang Sun, Hanqi Jin, Xiaojun Wan Wangxuan Institute of Computer Technology, Peking University Center for Data Science, Peking University The MOE Key Laboratory of Computational Linguistics, Peking University sunrenliangpku@gmail.com {jinhanqi, wanxiaojun}@pku.edu.cn Abstract Text simplification is a valuable technique. However, current research is limited to sen- tence simplification. In this paper, we define and investigate a new task of document-level text simplification, which aims to simplify a document consisting of multiple sentences. Based on Wikipedia dumps, we first construct a large-scale dataset named D-Wikipedia and perform analysis and human evaluation on it to show that the dataset is reliable. Then, we propose a new automatic evaluation met- ric called D-SARI that is more suitable for the document-level simplification task. Fi- nally, we select several representative mod- els as baseline models for this task and per- form automatic evaluation and human evalua- tion. We analyze the results and point out the shortcomings of the baseline models. 1 Introduction Text simplification is a valuable technique that de- serves to be studied in depth (Woodsend and La- pata, 2011). One definition of text simplification is to simplify the original text to a more under- standable text, while keeping the main meaning of the original text unchanged (Štajner and Saggion, 2018; Maddela et al., 2020). It can provide conve- nience for non-native speakers (Petersen and Osten- dorf, 2007; Glavaš and Štajner, 2015; Paetzold and Specia, 2016), non-expert readers (Elhadad and Sutaria, 2007; Siddharthan and Katsos, 2010) and children (De Belder and Moens, 2010; Kajiwara et al., 2013). 1.1 Why it is Valuable to Study Document-level Text Simplification Currently, researches on text simplification focus on sentence simplification, and the existing com- mon text simplification datasets such as Wikilarge, Wikismall, and Newsela are also designed for sen- tence simplification. However, various complex ap- plications in the real world often require document- level simplification rather than sentence-level sim- plification. Imagining that if you want to simplify an article in Time magazine for children to read, it is very inefficient to simplify the sentences sepa- rately. Besides, sentences that are obscure and have little relation to the subject should be deleted in- stead of simplified. Therefore, studying document- level text simplification may be more meaning- ful than studying sentence-level text simplification alone. Unfortunately, the research on document- level text simplification is still scarce: there is no formal definition, no suitable dataset, and evalua- tion criteria. 1.2 Similarities and Differences with Text Summarization Other tasks that may be related to document-level text simplification are text summarization(Dong et al., 2018; Cao et al., 2020), paraphrasing (Zhao et al., 2018; Guo et al., 2018), and split & rephrase (Narayan et al., 2017; Surya et al., 2019). Obvi- ously, the paraphrasing and split & rephrase tasks are both sentence-level tasks. The most closely related task is text summarization, which is also a document-level …

Content truncated to 500 words