Controllable Text Simplification with Explicit Paraphrasing

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3536–3553 June 6–11, 2021. ©2021 Association for Computational Linguistics 3536 Controllable Text Simplification with Explicit Paraphrasing Mounica Maddela1, Fernando Alva-Manchego2, Wei Xu1 1School of Interactive Computing, Georgia Institute of Technology 2Department of Computer Science, University of Sheffield {mounica.maddela, wei.xu}@cc.gatech.edu f.alva@sheffield.ac.uk Abstract Text Simplification improves the readability of sentences through several rewriting trans- formations, such as lexical paraphrasing, dele- tion, and splitting. Current simplification sys- tems are predominantly sequence-to-sequence models that are trained end-to-end to perform all these operations simultaneously. However, such systems limit themselves to mostly delet- ing words and cannot easily adapt to the re- quirements of different target audiences. In this paper, we propose a novel hybrid approach that leverages linguistically-motivated rules for splitting and deletion, and couples them with a neural paraphrasing model to produce varied rewriting styles. We introduce a new data augmentation method to improve the para- phrasing capability of our model. Through au- tomatic and manual evaluations, we show that our proposed model establishes a new state-of- the art for the task, paraphrasing more often than the existing systems, and can control the degree of each simplification operation applied to the input texts.1 1 Introduction Text Simplification aims to improve the readability of texts with simpler grammar and word choices while preserving meaning (Saggion, 2017). It pro- vides reading assistance to children (Kajiwara et al., 2013), non-native speakers (Petersen and Osten- dorf, 2007; Pellow and Eskenazi, 2014; Paetzold, 2016), and people with reading disabilities (Rello et al., 2013). It also helps with downstream nat- ural language processing tasks, such as parsing (Chandrasekar et al., 1996), semantic role labelling (Vickrey and Koller, 2008), information extraction (Miwa et al., 2010), and machine translation (MT, Chen et al., 2012; Štajner and Popovic, 2016). Since 2016, nearly all text simplification sys- tems have been sequence-to-sequence (seq2seq) 1Our code and data are available at https://github. com/mounicam/controllable_simplification. OLen %new %eq %split Complex (input) 20.7 0.0 100.0 0.0 Narayan and Gardent (2014)† 10.4 0.7 0.8 0.4 Zhang and Lapata (2017)† 13.8 8.1 16.8 0.0 Dong et al. (2019)† 10.9 8.4 4.6 0.0 Kriz et al. (2019)† 10.8 11.2 1.2 0.0 LSTM 17.0 6.1 28.4 1.2 Our Model 17.1 17.0 3.0 31.8 Simple (reference) 17.9 29.0 0.0 30.0 Table 1: Output statistics of 500 random sentences from the Newsela test set. Existing systems rely on deletion and do not paraphrase well. OLen, %new, %eq and %split denote the average output length, per- centage of new words added, percentage of system out- puts that are identical to the inputs, and percentage of sentence splits, respectively. †We used the system out- puts shared by their authors. models trained end-to-end, which have greatly in- creased the fluency of the outputs (Zhang and Lap- ata, 2017; Nisioi et al., 2017; Zhao et al., 2018; Kriz et al., 2019; Dong et al., 2019; Jiang et al., 2020). However, these systems mostly rely on deletion and tend to …

Content truncated to 500 words