CombiNMT: An Exploration into Neural Text Simplification Models

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5588–5594 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5588 CombiNMT: An Exploration into Neural Text Simplification Models Michael Cooper, Matthew Shardlow Department of Computing and Mathematics Manchester Metropolitan University mikejcooper90@gmail.com, m.shardlow@mmu.ac.uk Abstract This work presents a replication study of Exploring Neural Text Simplification Models (Nisioi et al., 2017). We were able to successfully replicate and extend the methods presented in the original paper. Alongside the replication results, we present our improvements dubbed CombiNMT. By using an updated implementation of OpenNMT, and incorporating the Newsela corpus alongside the original Wikipedia dataset (Hwang et al., 2016), as well as refining both datasets to select high quality training examples. Our work present two new systems, CombiNMT995, which is a result of matched sentences with a cosine similarity of 0.995 or less, and CombiNMT98, which, similarly, runs on a cosine similarity of 0.98 or less. By extending the human evaluation presented within the original paper, increasing both the number of annotators and the number of sentences annotated, with the intention of increasing the quality of the results, CombiNMT998 shows significant improvement over any of the Neural Text Simplification (NTS) systems from the original paper in terms of both the number of changes and the percentage of correct changes made. Keywords: Text Simplification, Replication, Neural Machine Translation 1. Introduction Neural Machine Translation (NMT) has brought significant improvements to the field of Natural Language Processing (NLP) especially when results from these systems are com- pared to those from Rule Based Statistical Machine Trans- lation (Weiss, 1995), Statistical Machine Translation Sys- tems (Hwang et al., 2015) and Neural Phrase Based Ma- chine Translation (Wu et al., 2016). NMT has become a widely used technique in Machine Translation (MT), as well as a well-regarded approach for other tasks, including dialogue generation, parsing and summarization. This paper is a replication study of the Neural Text Sim- plification (NTS) system by Nisioi et al. (2017) as part of the REPROLANG shared task at LREC 2020. As such, we employed the same methods and configurations as the orig- inal paper. We have built on this replication by employing an extended version of the human evaluation described in the original paper, as well as employing more recent tech- nologies to improve on the results from the original paper. In addition to the technologies used in the original pa- per, we refined the original corpus, using RoBERTa’s large model (Liu et al., 2019), finetuned on MNLI 1, to run the Semantic Text Similarity function as well as introducing further parallel data from Newsela derived in the same way. We also removed the sentences which were the most simi- lar, ensuring the system preferred more bold rewrites, learnt from the differences found in the sentences. 2. Background 2.1. Neural Text Simplification This paper is a replication study of Nisioi et al. (2017), in which the research team presented an Automated Text Simplification (ATS) system to …

Content truncated to 500 words