Multiattentive Recurrent Neural Network Architecture for Multilingual Readability Assessment

Multiattentive Recurrent Neural Network Architecture for Multilingual Readability Assessment Ion Madrazo Azpiazu and Maria Soledad Pera Department of Computer Science Boise State University {ionmadrazo,solepera}@boisestate.edu Abstract We present a multiattentive recurrent neural network architecture for automatic multilin- gual readability assessment. This architecture considers raw words as its main input, but internally captures text structure and informs its word attention process using other syntax- and morphology-related datapoints, known to be of great importance to readability. This is achieved by a multiattentive strategy that allows the neural network to focus on spe- cific parts of a text for predicting its reading level. We conducted an exhaustive evaluation using data sets targeting multiple languages and prediction task types, to compare the pro- posed model with traditional, state-of-the-art, and other neural network strategies. 1 Introduction For decades, readability assessment has been used by diverse stakeholders–from educators to public institutions—for determining the complexity of texts (Benjamin, 2012). Traditional formulas do so by focusing only on superficial linguistic features (e.g., average length of sentences or syllables per word). This leads to criticism, as these formulas do not explore deeper levels of text processing and thus yield rough estimates of complexity (i.e., difficulty) that often lack accuracy (Arf´e et al., 2018). In fact, traditional formulas can label a text as ‘‘easy to read’’ even if its content is completely nonsensical (Davison and Kantor, 1982). To improve the quality of automatic read- ability assessment, researchers turned to more sophisticated techniques that go beyond examin- ing shallow features. These techniques, typically based on supervised machine learning, incorpo- rate hundreds (even thousands) of features that describe a text from multiple perspectives: syntax, morphology, cohesion, discourse structure, and subject matter (Dell’Orletta et al., 2011; Franc¸ois and Fairon, 2012; Denning et al., 2016; Arf´e et al., 2018). The dependency on these numerous features, however, has made readability assess- ment tools too complex to deploy and apply to languages beyond the one for which they were originally designed. Furthermore, feature and language dependency, along with lack of homo- geneity in terms of readability scales, often prevent researchers from comparing new strate- gies with state-of-the-art counterparts, preventing community consensus on which features are the most beneficial for capturing text complexity (De Clercq and Hoste, 2016). Existing literature reflects the fact that appli- cations that leverage text complexity analysis, including book recommendation or categorization (Lexile, 2016; Pera and Ng, 2014), Web result summarization (Kanungo and Orr, 2009), and accessibility in the health domain (Bernstam et al., 2005; Fitzsimmons et al., 2010), still favor less precise but easier to implement alternatives, with Flesch as the most accepted choice (Ballesteros- Pe˜na and Fern´andez-Aedo, 2013; Bea-Mu˜noz et al., 2015). We argue that this is caused by the uncertainty induced by the lack of uniformity of readability scales, adaptability among readability assessment tools, and benchmarks. Areas of study that were historically heavily dependent on feature engineering, including sen- timent analysis or image processing (Manjunath and Ma, 1996; Abbasi et al., 2008), have made their way towards alternatives that do not …

Content truncated to 500 words