A New Approach to Automated Text Readability Classification based on Concept Indexing with Integrated Part-of-Speech n-gram Features

Proceedings of Recent Advances in Natural Language Processing, pages 521–528, Hissar, Bulgaria, Sep 7–9 2015. A New Approach to Automated Text Readability Classification based on Concept Indexing with Integrated Part-of-Speech n-gram Features Abigail R. Razon University of Birmingham ARR125@cs.bham.ac.uk John A. Barnden University of Birmingham J.A.BARNDEN@cs.bham.ac.uk Abstract This study is about the development of a learner-focused text readability indexing tool for second language learners (L2) of English. Student essays are used to cali- brate the system, making it capable of pro- viding realistic approximation of L2s’ ac- tual reading ability spectrum. The system aims to promote self-directed (i.e. self- study) language learning and help even those L2s who can not afford formal ed- ucation. In this paper, we provide a comparative re- view of two vectorial semantics-based al- gorithms, namely, Latent Semantic Index- ing (LSI) and Concept Indexing (CI) for text content analysis. Since these algo- rithms rely on the bag-of-words approach and inherently lack grammar-related anal- ysis, we augment them by incorporating Part-of-Speech (POS) n-gram features to approximate syntactic complexity of the text documents. Based on the results, CI-based features outperformed LSI-based features in most of the experiments. Without the integra- tion of POS n-gram features, the differ- ence between their mean exact agreement accuracies (MEAA) can reach as high as 23%, in favor of CI. It has also been proven that the performance of both algo- rithms can be further enhanced by combin- ing POS bi-gram features, yielding as high as 95.1% and 91.9% MEAA values for CI and LSI, respectively. 1 Introduction Text Readability is often defined as how easily documents can be read and understood. Moreover, Text Readability Indexing (TRI) is the process wherein texts are classified according to their dif- ficulty level based on educational standards set by institutions. We take it as one of our working hypotheses that language learning is something personal and that text interpretations are greatly influenced by the learner’s personality, preferences, experiences, and beliefs which are not something that can be easily set to a particular standard. Thus, TRI sys- tems should be modelled from the learners them- selves and that these systems should have the abil- ity to adapt to the learner’s learning progression. Past researches on this domain, such as (Si and Callan, 2001) and (Heilman et al., 2007), rely greatly on syntactic features as indicators of text readability. Such features include sentence length, syllable and character counts per word, part-of-speech (POS) tags, and word frequency. Although these features are important linguistic components, these have not been sufficient to model reading difficulty levels. As a result, recent studies are geared towards using content learning techniques from the Natural Language Processing (NLP) area. Such techniques include Latent Se- mantic Indexing (LSI) and Concept Indexing (CI) which have the ability to extract text content fea- tures that can be used to model learner profiles within each school grade level. There have been several attempts to combine grammar- and content-based features in readabil- ity analysis. However, it still hasn’t been fully …

Content truncated to 500 words