Coh-Metrix-Esp: A Complexity Analysis Tool for Documents Written in Spanish

Coh-Metrix-Esp: A Complexity Analysis Tool for Documents Written in Spanish Andre Quispesaravia, Walter Perez, Marco Sobrevilla Cabezudo, Fernando Alva-Manchego Research Group on Pattern Recognition and Applied Artificial Intelligence (GRPIAA) Pontificia Universidad Cat´olica del Per´u Av. Universitaria 1801, San Miguel, Lima 32, Peru {andre.quispesaravia, wperez, f.alva}@pucp.pe, msobrevilla@pucp.edu.pe Abstract Text Complexity Analysis is an useful task in Education. For example, it can help teachers select appropriate texts for their students according to their educational level. This task requires the analysis of several text features that people do mostly manually (e.g. syntactic complexity, words variety, etc.). In this paper, we present a tool useful for Complexity Analysis, called Coh-Metrix-Esp. This is the Spanish version of Coh-Metrix and is able to calculate 45 readability indices. We analyse how these indices behave in a corpus of “simple” and “complex” documents, and also use them as features in a complexity binary classifier for texts in Spanish. After some experiments with machine learning algorithms, we got 0.9 F-measure for a corpus that contains tales for kids and adults and 0.82 F-measure for a corpus with texts written for students of Spanish as a foreign language. Keywords: Complexity Analysis, Readability Assessment Indices, Coh-Metrix, Natural Language Processing 1. Introduction Knowledge is transmitted orally, in writing and through media. As such, reading is one of the most useful tools in the learning process, since it is our reading capability the one that let us access all this information (PISA, 2009). Three interrelated components assess the complexity in the reading comprehension process: i) qualitative dimension, where complexity is assessed by the meaning or purpose of the text; ii) quantitative dimension, where indices related to quantitative aspects of the texts (word length, frequency, incidence of grammar structures, etc.) are used to assess complexity; and iii) reader-task, where variables related to the reader, like motivation and knowledge of the task, are considered (Fisher et al., 2012). Qualitative and reader-task assessments require human readers, professional judgment and experience. However, quantitative assessment can be automated, thereby giving the opportunity to explore linguistic features and analyse how they reflect the complexity of the text. For quantitative assessment of the complexity of texts writ- ten in Spanish, there exist superficial formulae, like Flesh- Fernandez (Fern´andez Huerta, 1959), that classify texts de- pending on the score given by the formula. The informa- tion produced by this approach is limited and does not de- tect subtle changes related to the coherence and cohesion of the texts, which are factors related to its structure and the mental image formed as a representation of the texts by the reader (Graesser et al., 2004). In this paper, we focus on the quantitative dimension of text complexity and analyze how readability indices behave in text complexity assessment. We adapted 45 Coh-metrix indices (Section 3.) to Spanish integrating different natu- ral language processing (NLP) resources (Section 3.1.). To validate the indices, we present an analysis of text com- plexity for texts written in Spanish (Section 4.). The corpus used in this analysis is composed …

Content truncated to 500 words