Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification

Proceedings of NAACL-HLT 2019, pages 3137–3147 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3137 Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification Reno Kriz∗, Jo˜ao Sedoc∗, Marianna Apidianaki△, Carolina Zheng∗, Gaurav Kumar∗, Eleni Miltsakaki†, and Chris Callison-Burch∗ ∗ Computer and Information Science Department, University of Pennsylvania △ LIMSI, CNRS, Universit´e Paris-Saclay, 91403 Orsay & LLF, Univ. Paris Diderot † Choosito, Inc. {rekriz,joao,gauku,carzheng,ccb}@seas.upenn.edu, marianna@limsi.fr, eleni@choosito.com Abstract Sentence simplification is the task of rewriting texts so they are easier to understand. Recent research has applied sequence-to-sequence (Seq2Seq) models to this task, focusing largely on training-time improvements via reinforce- ment learning and memory augmentation. One of the main problems with applying generic Seq2Seq models for simplification is that these models tend to copy directly from the origi- nal sentence, resulting in outputs that are rel- atively long and complex. We aim to alle- viate this issue through the use of two main techniques. First, we incorporate content word complexities, as predicted with a leveled word complexity model, into our loss function dur- ing training. Second, we generate a large set of diverse candidate simplifications at test time, and rerank these to promote fluency, adequacy, and simplicity. Here, we measure simplicity through a novel sentence complexity model. These extensions allow our models to per- form competitively with state-of-the-art sys- tems while generating simpler sentences. We report standard automatic and human evalua- tion metrics.1 1 Introduction Automatic text simplification aims to reduce the complexity of texts and preserve their meaning, making their content more accessible to a broader audience (Saggion, 2017). This process can ben- efit people with reading disabilities, foreign lan- guage learners and young children, and can assist non-experts exploring a new field. Text simplifica- tion has gained wide interest in recent years due to its relevance for NLP tasks. Simplifying text dur- ing preprocessing can improve the performance of syntactic parsers (Chandrasekar et al., 1996) and 1Our code is available in our fork of Sockeye (Hieber et al., 2017) at https://github.com/rekriz11/sockeye-recipes. Figure 1: Example comparison of a simplification gen- erated by a standard Seq2Seq model vs. our model. semantic role labelers (Vickrey and Koller, 2008; Woodsend and Lapata, 2014), and can improve the grammaticality (fluency) and meaning preserva- tion (adequacy) of translation output (ˇStajner and Popovic, 2016). Most text simplification work has approached the task as a monolingual machine translation problem (Woodsend and Lapata, 2011; Narayan and Gardent, 2014). Once viewed as such, a natural approach is to use sequence-to-sequence (Seq2Seq) models, which have shown state-of- the-art performance on a variety of NLP tasks, in- cluding machine translation (Vaswani et al., 2017) and dialogue systems (Vinyals and Le, 2015). One of the main limitations in applying stan- dard Seq2Seq models to simplification is that these models tend to copy directly from the original complex sentence too often, as this is the most common operation in simplification. Several re- cent efforts have attempted to alleviate this prob- lem using reinforcement learning (Zhang and Lap- ata, 2017) and …

Content truncated to 500 words