An Automatic Tool For Language Evaluation

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 1721–1726 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 1721 An Automatic Tool For Language Evaluation Fabio Fassetti, Ilaria Fassetti DIMES - University of Calabria, Therapeia Rehabilitation Center fabio.fassetti@unical.it, ilaria.fassetti@gmail.com Abstract The aim of evaluating children speech and language is to measure their communication skills. In particular, the speech language patholo- gist is interested in determining the child’s impairments in the areas of language, articulation, voice, fluency and swallowing. In literature some standardized tests have been proposed to assess and screen developmental language impairments but they require manual labo- rious transcription, annotation and calculation. This work is very time demanding and, also, may introduce several kinds of errors in the evaluation phase and non-uniform evaluations. In order to help therapists, a system performing automated evaluation is proposed. Providing as input the correct sentence and the sentence produced by patients, the technique evaluates the level of the verbal production and returns a score. The main phases of the method concern an ad-hoc transformation of the produced sentence in the reference sentence and in the evaluation of the cost of this transformation. Since the cost function is related to many weights, a learning phase is defined to automatically set such weights. Keywords: NLP, Language Evaluation, Machine Learning 1. Introduction The purpose of a speech and language evaluation is to measure communication skills specially of children. The speech language pathologist is interested in determining the child’s impairments in the following main areas: language, articulation, voice, fluency, swallowing. In this work sev- eral components of the human language are taken into consideration, namely phonology, morphology and syntax. Phonology is the speech sound system of language and the rules for how speech sounds are combined. Morphology is the study of of word structure and of rules that govern how morphemes (the smallest meaningful units of language) are used in a language. Syntax is the study of rules that gov- ern the ways words combine to form phrases, clauses and sentences. In this context, standardized tests, like the highly diffused tests (Devescovi and Caselli, 2001; Vender et al., 1981; Cianchetti and Fancello, 1997) for Italian speakers, are reg- ularly used for assessing and screening developmental lan- guage impairments but require manual laborious transcrip- tion, annotation and calculation. This meticulous work of- ten leads to make mistakes and to spend too much time. Here, a system performing automated evaluation is pro- posed. It evaluates the different components of the human language (phonology,morphology, syntax) included the de- termination of phonological processes, both structural and systemic, through a sentence repetition task. Differently from the system proposed in (Fassetti and Fassetti, 2018), where reading abilities are tested by asking the patient to rewrite read texts, here phonologic and language production abilities are tested and the patient does not directly produce written text but speaks and the therapist transcribe produced sentences to feed the system. Sentence repetition tasks are used in the evaluation …

Content truncated to 500 words