Zero-Shot Crosslingual Sentence Simplification

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 5109–5126, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 5109 Zero-Shot Crosslingual Sentence Simplification Jonathan Mallinson1 Rico Sennrich2,1 Mirella Lapata1 1School of Informatics, University of Edinburgh 2Department of Computational Linguistics, University of Zurich J.Mallinson@ed.ac.uk, Sennrich@cl.uzh.ch, mlap@inf.ed.ac.uk Abstract Sentence simplification aims to make sen- tences easier to read and understand. Re- cent approaches have shown promising results with encoder-decoder models trained on large amounts of parallel data which often only ex- ists in English. We propose a zero-shot mod- eling framework which transfers simplifica- tion knowledge from English to another lan- guage (for which no parallel simplification corpus exists) while generalizing across lan- guages and tasks. A shared transformer en- coder constructs language-agnostic representa- tions, with a combination of task-specific en- coder layers added on top (e.g., for translation and simplification). Empirical results using both human and automatic metrics show that our approach produces better simplifications than unsupervised and pivot-based methods. 1 Introduction Sentence simplification aims to reduce the linguis- tic complexity of a text whilst retaining most of its meaning. It has been the subject of several mod- eling efforts in recent years due to its relevance to various applications (Siddharthan, 2014; Shard- low, 2014). Examples include the development of reading aids for individuals with autism (Evans et al., 2014), aphasia (Carroll et al., 1999), dyslexia (Rello et al., 2013), and population groups with low-literacy skills (Watanabe et al., 2009), such as children and non-native speakers. Modern approaches (Zhang and Lapata, 2017; Mallinson and Lapata, 2019; Nishihara et al., 2019; Dong et al., 2019) view the simplification task as monolingual text-to-text rewriting and employ the very successful encoder-decoder neural architec- ture (Bahdanau et al., 2015; Sutskever et al., 2014). In contrast to traditional methods, which target in- dividual aspects of the simplification task, such as sentence splitting (Carroll et al. 1999; Chan- drasekar et al. 1996, inter alia) or the substitution of complex words with simpler ones (Devlin, 1999; Kaji et al., 2002), neural models have no special purpose mechanisms for ensuring how to best sim- plify text. They rely on representation learning to implicitly capture simplification rewrites from data, i.e., examples of complex-simple sentence pairs. While large-scale parallel datasets exist for En- glish (Xu et al., 2015; Zhang and Lapata, 2017) and Spanish (Agrawal and Carpuat, 2019), there is a limited amount of simplification data for other languages. For example, Klaper et al. (2013) auto- matically aligned 7,000 complex-simple German sentences,1 and Brunato et al. (2015) released 1,000 complex-simple Italian sentences. But data- driven approaches to simplification, in particular popular neural models, require significantly more training data to achieve good performance, making these datasets better suited for testing or develop- ment purposes. Unsupervised approaches (Surya et al., 2019; Artetxe et al., 2018) which forgo the use of parallel corpora are an appealing solution to overcoming the paucity of data. However, in this paper we argue that better simplification mod- els can be obtained by taking advantage …

Content truncated to 500 words