Lexical Quantile-Based Text Complexity Measure

Proceedings of Recent Advances in Natural Language Processing, pages 270–275, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_031 270 Lexical Quantile-Based Text Complexity Measure Maksim Eremeev AITHEA m.eremeev@aithea.com Konstantin Vorontsov National University of Science and Technology MISIS voron@aithea.com Abstract This paper introduces a new approach to esti- mating the text document complexity. Com- mon readability indices are based on average length of sentences and words. In contrast to these methods, we propose to count the num- ber of rare words occurring abnormally often in the document. We use the reference corpus of texts and the quantile approach in order to determine what words are rare, and what fre- quencies are abnormal. We construct a general text complexity model, which can be adjusted for the specific task, and introduce two special models. The experimental design is based on a set of thematically similar pairs of Wikipedia articles, labeled using crowdsourcing. The ex- periments demonstrate the competitiveness of the proposed approach. 1 Introduction Automated text complexity measurement tools have been proposed in order to help teachers to select textbooks that correspond to the students’ comprehension level and publishers to explore whether their articles are readable. Thus, plenty of readability indexes were developed. Mea- sures like Automated Readability Index (Senter and Smith, 1967), Flesch-Kincaid readability tests (Flesh, 1951), SMOG index (McLaughlin, 1969), Gunning fog (Gunning, 1952) and etc. use heuris- tics based on simple statistics such as total number of words, mean number of words per sentence, to- tal number of sentences or even number of sylla- bles to evaluate how complex given text is. By combining these statistics with different weight- ing factors, readability indexes assign the given document a complexity score, which is, in most cases, the approximate representation of the US grade level needed to comprehend the text. For in- stance, an Automated Readability Index (ARI) has the following form for the document d: ARI(d) = 4.71 × c w + 0.5 × w s − 21.43 (1) where c refers to the total number of letters in the document d, w is the total number of words and s denotes the total number of sentences in d. Since readability indexes rely on a few basic factors, precise assessment requires aggregation of many scores. Thus, Coh-Metrix-PORT tool (Aluisio et al., 2010) includes more than 50 dif- ferent indexes for Portuguese language. The tool is based on Coh-Metrix (Graesser et al., 2004) principles to estimate complexity and cohesion not only for explicit text, but for the mental represen- tation of the document. Readability indexes are interpretable and easy to implement. However, the great number of con- stants tuned specifically for the English language texts, lack of the semantics consideration and tai- loring to the US grade level system restrains the number of possible applications. As for the non-English languages, several lex- ical and morphological features for Italian to solve text simplification problem were presented (Brunato et al., 2015), supervised approach in readability estimations was introduced (vor der Brck et al., 2008) and the …

Content truncated to 500 words