FABRA: French Aggregator-Based Readability Assessment toolkit

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 1217–1233 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 1217 FABRA: French Aggregator-Based Readability Assessment toolkit Rodrigo Wilkens∗, David Alfter∗, Xiaoou Wang∗, Alice Pintard∗, Ana¨ıs Tack†, Kevin Yancey⋄, Thomas Franc¸ois∗ ∗Cental, IL&C, UCLouvain, †Stanford University, ⋄Duolingo {rodrigo.wilkens, david.alfter, xiaoou.wang, alice.pintard}@uclouvain.be, atack@stanford.edu, kyancey@duolingo.com, thomas.francois@uclouvain.be Abstract In this paper, we present the FABRA readability toolkit based on the aggregation of a large number of readability predictor variables. The toolkit is implemented as a service-oriented architecture, which obviates the need for installation, and simplifies its integration into other projects. We also perform a set of experiments to show which features are most predictive on two different corpora, and how the use of aggregators improves performance over standard feature-based readability prediction. Our experiments show that, for the explored corpora, the most important predictors for native texts are measures of lexical diversity and dependency counts while the most important ones for foreign texts are syntactic variables illustrating language development, as well as features linked to lexical gradation in FFL textbooks. FABRA has the potential to support new research on readability assessment for French. Keywords: readability, French, readability score encoding, automatic readability assessment 1. Introduction International surveys on reading abilities such as PISA (Schleicher, 2019) regularly remind us that about 20% of the 15-year-old students can be considered as poor readers. Reading deficiencies can have severe conse- quences on people’s lives, such as the inability to ac- cess crucial information (e.g., medical data) (Friedman and Hoffman-Goetz, 2006), to process administrative applications (Kimble, 1992), or to find a job. This chal- lenge has been addressed in a variety of ways, as the development of methods to assess reading difficulty of texts for a given audience within the field of readability. The readability research can be traced back to the 1920s’, with the seminal work of Lively and Pressey (1923) that originated a tradition of statistical models aiming to predict the reading difficulty of texts, called readability formulas.1 The evolution of readability for- mulas across the 20th century can be roughly summa- rized as follows: initially, readability formulas were computed by hand and were therefore designed as a trade-off between reliability and minimization of ef- fort. The most famous ones combine two textual fea- tures, such as syllable count (Flesch, 1948), sentence length (Flesch, 1948; Dale and Chall, 1948), or pro- portion of easy words (Dale and Chall, 1948). Later, with the dawn of computers, the first automatized for- mulas appeared, such as the Automated Readability In- dex (Smith and Senter, 1967). In addition, readabil- ity formulas included more features (Bormuth, 1966; Coleman and Liau, 1975) that mostly remained surface features in the 70s, then evolved to capture more so- phisticated characteristics of texts, such as coherence or inference load (Kintsch and Vipond, 1979). For the rest of the 20th, the statistical models used to design readability formulas hardly evolved, whereas innova- 1Readability cannot be confused with Text Simplification that …

Content truncated to 500 words