A Syntax-Aware Edit-based System for Text Simplification

Proceedings of Recent Advances in Natural Language Processing, pages 324–334 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_038 324 A Syntax-Aware Edit-based System for Text Simplification Oscar M. Cumbicus-Pineda, Ixa group and Carrera de Computación UPV/EHU and UNL ocumbicus001@ikasle.ehu.es Itziar Gonzalez-Dios, Aitor Soroa Ixa group, HiTZ center University of the Basque Country (UPV/EHU) {itziar.gonzalezd,a.soroa}@ehu.eus Abstract Edit-based text simplification systems have at- tained much attention in recent years due to their ability to produce simplification solu- tions that are interpretable, as well as requir- ing less training examples compared to tradi- tional seq2seq systems. Edit-based systems learn edit operations at a word level, but it is well known that many of the operations per- formed when simplifying text are of a syntac- tic nature. In this paper we propose to add syntactic information into a well known edit- based system. We extend the system with a graph convolutional network module that mim- ics the dependency structure of the sentence, thus giving the model an explicit representa- tion of syntax. We perform a series of experi- ments in English, Spanish and Italian, and re- port improvements of the state of the art in four out of five datasets. Further analysis shows that syntactic information is always beneficial, and suggest that syntax is more helpful in com- plex sentences. 1 Introduction Automatic Text Simplification (ATS) aims to re- duce the linguistic complexity of a text for a cer- tain target audience. ATS is useful for people learning foreign languages, suffering cognitive dis- abilities, or with low literacy level. The area of application of ATS is however not restricted to humans, as it has also been used to improve au- tomatic NLP tasks such as parsing (Chandrasekar et al., 1996a), summarization (Beigman Klebanov et al., 2004; Silveira and Branco, 2012), semantic role labeling (Vickrey and Koller, 2008; Woodsend and Lapata, 2017), information extraction (Evans and Orasan, 2019) and machine translation (Gerber and Hovy, 1998; Štajner and Popovic, 2016; Hasler et al., 2017), among others. ATS is often cast as a machine translation task, where the system receives complex sentences as input, and produces sentences that are simpler yet maintaining the original meaning. While early ATS systems were rule-based, statistical or hybrid (Sag- gion, 2017), in the last years neural network based ATS approaches have also been proposed (Alva- Manchego et al., 2020). In particular, sequence-to- sequence (seq2seq) neural models (Sutskever et al., 2014; Nisioi et al., 2017; Zhang and Lapata, 2017a) have shown to obtain state-of-the-art results. Such systems are trained on parallel corpora comprising pairs of complex/simple sentences, and implicitly learn the simplification rewrites needed to convert complex sentences into simpler ones. Neural seq2seq systems are usually black boxes that are trained on an end-to-end fashion. As a consequence, built models are usually very diffi- cult to interpret, and offer little control or hints that explain why a particular input word sequence has been rephrased. Edit-based ATS systems try to overcome this limitation by learning the trans- formations required to convert complex sentences into their simpler counterparts. …

Content truncated to 500 words