Readability Indices for Automatic Evaluation of Text Simplification Systems: A Feasibility Study for Spanish

International Joint Conference on Natural Language Processing, pages 374–382, Nagoya, Japan, 14-18 October 2013. Readability Indices for Automatic Evaluation of Text Simplification Systems: A Feasibility Study for Spanish Sanja ˇStajner Research Group in Computational Linguistics University of Wolverhampton, UK sanjastajner@wlv.ac.uk Horacio Saggion TALN Research Group Universitat Pompeu Fabra, Spain horacio.saggion@upf.edu Abstract This paper addresses the problem of automatic evaluation of text simplifica- tion systems for Spanish. We test whether already-existing readability for- mulae would be suitable for this task. We adapt three existing readability indices (two measuring lexical complexity and one measuring syntactic complexity) to be computed automatically, which are then applied to a corpus of original news texts and their manual simplifications aimed at people with cognitive disabilities. We show that there is a significant correlation between each of the three readability in- dices and several linguistically motivated features which might be seen as reading obstacles for various target populations. Furthermore, we show that there is a sig- nificant correlation between the two read- ability indices which measure lexical com- plexity. 1 Introduction In recent years, there has been growing effort to simplify written material and make it equally ac- cessible to everyone. Various studies indicate that lexically and syntactically complex texts can be very difficult for non-native speakers and people with various reading impairments (e.g. autistic, aphasic, dyslexic or deaf people). Aphasic peo- ple, for instance, may encounter problems with less frequent words and some particular sentence constructions (Devlin, 1999). They also have problems in understanding syntactic constructions which do not follow the canonical subject-verb- object structure (e.g. passive constructions), and especially those sentences which are semantically reversible, e.g. “The boy was kissed by the girl” (Carroll et al., 1999). Additionally, apha- sic readers may have additional problems with comprehending newswire texts which have some genre-specific characteristics. These types of texts tend to use long sentences, noun compounds and long sequences of adjectives, e.g. “Twenty- five-year-old blonde-haired mother-of-two Jane Smith” (Carroll et al., 1999). People with intellec- tual disabilities have problem with both lexically and syntactically complex texts, as well as with processing and loading large amounts of informa- tion (Feng, 2009). Since the late nineties, several initiatives which proposed guidelines for producing plain, easy- to-read and more accessible documents have emerged, e.g. ”The Plain Language Action and Information Network (PLAIN)”1, “Make it Sim- ple, European Guidelines for the Production of Easy-to-Read Information for people with Learn- ing Disability” (Freyhoff et al., 1998), “Am I mak- ing myself clear? Mencap’s guidelines for ac- cessible writing”2, and “Web content accessibil- ity guidelines”3. All these guidelines share sim- ilar instructions for accessible writing, some of them more detailed than others. They all advise the writer to use the active voice instead of pas- sive, use the simplest form of a verb (present and not conditional or future), avoid hidden verbs (i.e. verbs converted into a noun), use short, simple words and omit unnecessary words, write short sentences and cover only one main idea per sen- tence, etc. Armed with these …

Content truncated to 500 words