EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3393–3402 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3393 EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing Yue Dong1, Zichao Li2, Mehdi Rezagholizadeh2, Jackie Chi Kit Cheung1 1MILA, McGill yue.dong2@mail.mcgill.ca, jcheung@cs.mcgill.ca 2Huawei Noah’s Ark Lab {li.zichao, mehdi.rezagholizadeh}@huawei.com Abstract We present the first sentence simplification model that learns explicit edit operations (ADD, DELETE, and KEEP) via a neural programmer-interpreter approach. Most cur- rent neural sentence simplification systems are variants of sequence-to-sequence models adopted from machine translation. These methods learn to simplify sentences as a byproduct of the fact that they are trained on complex-simple sentence pairs. By con- trast, our neural programmer-interpreter is di- rectly trained to predict explicit edit operations on targeted parts of the input sentence, re- sembling the way that humans might perform simplification and revision. Our model out- performs previous state-of-the-art neural sen- tence simplification models (without external knowledge) by large margins on three bench- mark text simplification corpora in terms of SARI (+0.95 WikiLarge, +1.89 WikiSmall, +1.41 Newsela), and is judged by humans to produce overall better and simpler output sen- tences1. 1 Introduction Sentence simplification aims to reduce the read- ing complexity of a sentence while preserving its meaning. Simplification systems can bene- fit populations with limited literacy skills (Watan- abe et al., 2009), such as children, second lan- guage speakers and individuals with language im- pairments including dyslexia (Rello et al., 2013), aphasia (Carroll et al., 1999) and autism (Evans et al., 2014). Inspired by the success of machine transla- tion, many text simplification (TS) systems treat sentence simplification as a monolingual transla- tion task, in which complex-simple sentence pairs 1Link to our code and data can be found here https: //github.com/yuedongP/EditNTS. are presented to the models as source-target pairs (Zhang and Lapata, 2017). Two major machine translation (MT) approaches are adapted into TS systems, each with its advantages: statistical ma- chine translation (SMT)-based models (Zhu et al., 2010; Wubben et al., 2012; Narayan and Gar- dent, 2014; Xu et al., 2016) can easily integrate human-curated features into the model, while neu- ral machine translation (NMT)-based models (Ni- sioi et al., 2017; Zhang and Lapata, 2017; Vu et al., 2018) can operate in an end-to-end fashion by ex- tracting features automatically. Nevertheless, MT- based models must learn the simplifying opera- tions that are embedded in the parallel complex- simple sentences implicitly. These operations are relatively infrequent, as a large part of the origi- nal complex sentence usually remains unchanged in the simplification process (Zhang et al., 2017). This leads to MT-based models that often produce outputs that are identical to the inputs (Zhao et al., 2018), which is also confirmed in our experiments. We instead propose a novel end-to-end Neu- ral Programmer-Interpreter (Reed and de Freitas, 2016) that learns to explicitly generate edit opera- tions in a sequential fashion, resembling the way that a human editor might perform simplifications on sentences. Our …

Content truncated to 500 words