MUSS: Multilingual Unsupervised Sentence Simplification by Mining Paraphrases

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 1651–1664 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 1651 MUSS: Multilingual Unsupervised Sentence Simplification by Mining Paraphrases Louis Martin1,2 Angela Fan1,3 ´Eric de la Clergerie2 Antoine Bordes1 Benoˆıt Sagot2 1Meta AI Research (Paris), 2Inria (Paris), 3LORIA (Nancy) {louismartin, angelafan, abordes}@fb.com {eric.de la clergerie, benoit.sagot}@inria.fr Abstract Progress in sentence simplification has been hindered by a lack of labeled parallel simplification data, particularly in languages other than English. We introduce MUSS, a Multilingual Unsupervised Sentence Simplification system that does not require labeled simplification data. MUSS uses a novel approach to sentence simplification that trains strong models using sentence- level paraphrase data instead of proper simplification data. These models leverage unsupervised pretraining and controllable generation mechanisms to flexibly adjust attributes such as length and lexical complexity at inference time. We show that this paraphrase data can be mined in any language from Common Crawl using semantic sentence embeddings, thus removing the need for labeled data. We evaluate our approach on English, French, and Spanish simplification benchmarks and closely match or outperform the previous best supervised results, despite not using any labeled simplification data. We push the state of the art further by incorporating labeled simplification data. 1. Introduction Sentence simplification is the task of making a sen- tence easier to read and understand by reducing its lex- ical and syntactic complexity, while retaining most of its original meaning. Simplification has a variety of important societal applications, for example increas- ing accessibility for those with cognitive disabilities such as aphasia (Carroll et al., 1998), dyslexia (Rello et al., 2013), and autism (Evans et al., 2014), or for non-native speakers (Paetzold and Specia, 2016). Re- search has mostly focused on English simplification, where source texts and associated simplified texts ex- ist and can be automatically aligned, such as English Wikipedia and Simple English Wikipedia (Zhang and Lapata, 2017). However, such data is limited in terms of size and domain, and difficult to find in other lan- guages. Additionally, simplifying a sentence can be achieved in multiple ways, and depend on the target au- dience. Simplification guidelines are not uniquely de- fined, outlined by the stark differences in English sim- plification benchmarks (Alva-Manchego et al., 2020a). This highlights the need for more general models that can adjust to different simplification scenarios. In this paper, we propose to train controllable models using sentence-level paraphrase data only, i.e. paral- lel sentences that have the same meaning but phrased differently. In order to generate simplifications and not paraphrases at test time, we use ACCESS (Mar- tin et al., 2020) to control attributes such as length, lexical and syntactic complexity. Paraphrase data is more readily available, and opens the door to training flexible models that can adjust to more varied simpli- fication scenarios. Our original goal was to mine sim- plifications from the web, but we surprisingly discov- ered that mining paraphrases leads to controllable mod- els with better simplification performance …

Content truncated to 500 words