Neural Readability Pairwise Ranking for Sentences in Italian Administrative Language

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 849–866 November 20–23, 2022. ©2022 Association for Computational Linguistics 849 Neural Readability Pairwise Ranking for Sentences in Italian Administrative Language Martina Miliani1,2 and Serena Auriemma2 and Fernando Alva-Manchego3 and Alessandro Lenci2 1 University for Foreigners of Siena 2 CoLing Lab, Department of Philology, Literature, and Linguistics, University of Pisa 3 School of Computer Science and Informatics, Cardiff University, UK martina.miliani@fileli.unipi.it, serena.auriemma@phd.unipi.it alvamanchegof@cardiff.ac.uk , alessandro.lenci@unipi.it Abstract Automatic Readability Assessment aims at as- signing a complexity level to a given text, which could help improve the accessibility to information in specific domains, such as the administrative one. In this paper, we investi- gate the behavior of a Neural Pairwise Ranking Model (NPRM) for sentence-level readability assessment of Italian administrative texts. To deal with data scarcity, we experiment with cross-lingual, cross- and in-domain approaches, and test our models on Admin-It, a new par- allel corpus in the Italian administrative lan- guage, containing sentences simplified using three different rewriting strategies. We show that NPRMs are effective in zero-shot scenarios (∼0.78 ranking accuracy), especially with rank- ing pairs containing simplifications produced by overall rewriting at the sentence-level, and that the best results are obtained by adding in- domain data (achieving perfect performance for such sentence pairs). Finally, we investigate where NPRMs failed, showing that the charac- teristics of the data used for fine-tuning, rather than its size, have a bigger effect on a model’s performance. 1 Introduction Due to its complexity, the style of Italian admin- istrative texts has been defined as “artificial” and “obscure” (Lubello, 2014). During the last decades, Italian institutions have fostered the use of a plain language in writing official acts and communica- tions (Fortis, 2005). However, the readability of Italian administrative texts still remains an issue (Cortelazzo, 2021), and measuring their complex- ity can help institutions improve information ac- cessibility, and guarantee a substantive equality of citizens (Vedovelli and De Mauro, 1999). One way to tackle this problem is with tech- nologies for Automatic Readability Assessment (ARA) that predict the complexity of texts (Collins- Thompson, 2014). This task has been widely in- vestigated in the educational domain, usually clas- sifying texts according to school grade levels or international frameworks for language proficiency. Currently, most models for ARA are based on neu- ral networks (Vajjala, 2022), which are trained in a supervised fashion by fine-tuning pre-trained lan- guage models (Imperial, 2021; Martinc et al., 2021; Lee and Vajjala, 2022). However, this approach could require large amounts of monolingual in- domain data, which is limited in specific sectorial languages like the one used in Italian administrative texts, for which the available resources are quite scarce (Tonelli et al., 2016; Brunato, 2015). In this paper, we tackle the data scarcity issue in two ways. First, we introduce Admin-It (Sec. 3), a parallel corpus in the Italian administrative lan- guage with sentences that were …

Content truncated to 500 words