Using Eye-tracking Data to Predict the Readability of Brazilian Portuguese Sentences in Single-task, Multi-task and Sequential Transfer Learning Approaches

Proceedings of the 28th International Conference on Computational Linguistics, pages 5821–5831 Barcelona, Spain (Online), December 8-13, 2020 5821 Using Eye-tracking Data to Predict the Readability of Brazilian Portuguese Sentences in Single-task, Multi-task and Sequential Transfer Learning Approaches Sidney Evaldo Leal 1 sidleal@gmail.com Jo˜ao Marcos Munguba Vieira 2,3 joaomvieira@gmail.com Erica dos Santos Rodrigues 4 ericasr@puc-rio.br Elisˆangela Nogueira Teixeira 2,3 elisteixeira@letras.ufc.br Sandra Maria Alu´ısio 1 sandra@icmc.usp.br 1 Instituto de Ciˆencias Matem´aticas e de Computac¸˜ao - Universidade de S˜ao Paulo (USP) Av. do Trabalhador Saocarlense, 400, S˜ao Carlos - SP - Brazil 2 Programa de P´os-graduac¸˜ao em Lingu´ıstica - Universidade Federal do Cear´a (UFC) Avenida da Universidade, 2683, BL. 125, 1o andar - Fortaleza - CE - Brazil 3 Laborat´orio de Ciˆencias Cognitivas e Psicolingu´ıstica - Universidade Federal do Cear´a (UFC) Avenida da Universidade, 2683, BL. 125, Sala 4, 1o andar - Fortaleza - CE - Brazil 4 Programa de P´os-graduac¸˜ao em Estudos da Linguagem - Pontif´ıcia Universidade Cat´olica do Rio de Janeiro (PUC-Rio) Laborat´orio de Psicolingu´ıstica e Aquisic¸˜ao da Linguagem (LAPAL/ PUC-Rio) Rua Marquˆes de S˜ao Vicente, 225 - Edif´ıcio Pe. Leonel Franca, 3o andar - G´avea - Rio de Janeiro - RJ - Brazil Abstract Sentence complexity assessment is a relatively new task in Natural Language Processing. One of its aims is to highlight in a text which sentences are more complex to support the simplification of contents for a target audience (e.g., children, cognitively impaired users, non-native speakers and low-literacy readers (Scarton and Specia, 2018)). This task is evaluated using datasets of pairs of aligned sentences including the complex and simple version of the same sentence. For Brazilian Portuguese, the task was addressed by (Leal et al., 2018), who set up the first dataset to evaluate the task in this language, reaching 87.8% of accuracy with linguistic features. The present work advances these results, using models inspired by (Gonzalez-Gardu˜no and Søgaard, 2018), which hold the state-of-the-art for the English language, with multi-task learning and eye- tracking measures. First-Pass Duration, Total Regression Duration and Total Fixation Duration were used in two moments; first to select a subset of linguistic features and then as an auxiliary task in the multi-task and sequential learning models. The best model proposed here reaches the new state-of-the-art for Portuguese with 97.5% accuracy1, an increase of almost 10 points compared to the best previous results, in addition to proposing improvements in the public dataset after analysing the errors of our best model. 1 Introduction Readability is the ease of reading a text, not in its typographical aspects such as font size, but by measures such as its syntactic structure complexity, vocabulary frequency, content, style, and organisation that can be fitted to prior knowledge, reading skill, interest and motivation of the reader (Dubay, 2007). Tracking the automation of readability back to its origin, the first readability formulas can be found a century ago in the United States, aiming to help teachers, librarians and scholars to select reading material 1Accuracy in our task is how close the model is …

Content truncated to 500 words