Problems in Current Text Simplification Research: New Data Can Help

Problems in Current Text Simplification Research: New Data Can Help Wei Xu1 and Chris Callison-Burch1 and Courtney Napoles2 1 Computer and Information Science Department University of Pennsylvania {xwe, ccb}@seas.upenn.edu 2 Department of Computer Science Johns Hopkins University courtneyn@jhu.edu Abstract Simple Wikipedia has dominated simplifica- tion research in the past 5 years. In this opinion paper, we argue that focusing on Wikipedia limits simplification research. We back up our arguments with corpus analy- sis and by highlighting statements that other researchers have made in the simplification literature. We introduce a new simplifica- tion dataset that is a significant improvement over Simple Wikipedia, and present a novel quantitative-comparative approach to study the quality of simplification data resources. 1 Introduction The goal of text simplification is to rewrite com- plex text into simpler language that is easier to un- derstand. Research into this topic has many poten- tial practical applications. For instance, it can pro- vide reading aids for people with disabilities (Carroll et al., 1999; Canning et al., 2000; Inui et al., 2003), low-literacy (Watanabe et al., 2009; De Belder and Moens, 2010), non-native backgrounds (Petersen and Ostendorf, 2007; Allen, 2009) or non-expert knowledge (Elhadad and Sutaria, 2007; Siddharthan and Katsos, 2010). Text simplification may also help improve the performance of many natural language processing (NLP) tasks, such as parsing (Chan- drasekar et al., 1996), summarization (Siddharthan et al., 2004; Klebanov et al., 2004; Vanderwende et al., 2007; Xu and Grishman, 2009), semantic role labeling (Vickrey and Koller, 2008), information ex- traction (Miwa et al., 2010) and machine transla- tion (Gerber and Hovy, 1998; Chen et al., 2012), by transforming long, complex sentences into ones that are more easily processed. The Parallel Wikipedia Simplification (PWKP) corpus prepared by Zhu et al. (2010), has become the benchmark dataset for training and evaluating automatic text simplification systems. An associated test set of 100 sentences from Wikipedia has been used for comparing the state-of-the-art approaches. The collection of simple-complex parallel sentences sparked a major advance for machine translation- based approaches to simplification. However, we will show that this dataset is deficient and should be considered obsolete. In this opinion paper, we argue that Wikipedia as a simplification data resource is suboptimal for several reasons: 1) It is prone to automatic sentence align- ment errors; 2) It contains a large proportion of in- adequate simplifications; 3) It generalizes poorly to other text genres. These problems are largely due to the fact that Simple Wikipedia is an encyclopedia spontaneously and collaboratively created for “chil- dren and adults who are learning English language” without more specific guidelines. We quantitatively illustrate the seriousness of these problems through manual inspection and statistical analysis. Our manual inspection reveals that about 50% of the sentence pairs in the PWKP corpus are not sim- plifications. We also introduce a new comparative approach to simplification corpus analysis. In par- ticular, we assemble a new simplification corpus of news articles,1 re-written by professional editors to meet the readability standards for children at multi- 1This Newsela …

Content truncated to 500 words