Simple TICO-19: A Dataset for Joint Translation and Simplification of COVID-19 Texts

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 3093–3102 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 3093 Simple TICO-19: A Dataset for Joint Translation and Simplification of COVID-19 Texts Matthew Shardlow∗, Fernando Alva-Manchego† ∗Manchester Metropolitan University, †Cardiff University m.shardlow@mmu.ac.uk, alvamanchegof@cardiff.ac.uk Abstract Specialist high-quality information is typically first available in English, and it is written in a language that may be difficult to understand by most readers. While Machine Translation technologies contribute to mitigate the first issue, the translated content will most likely still contain complex language. In order to investigate and address both problems simultaneously, we introduce Simple TICO-19, a new language resource containing manual simplifications of the English and Spanish portions of the TICO-19 corpus for Machine Translation of COVID-19 literature. We provide an in-depth description of the annotation process, which entailed designing an annotation manual and employing four annotators (two native English speakers and two native Spanish speakers) who simplified over 6,000 sentences from the English and Spanish portions of the TICO-19 corpus. We report several statistics on the new dataset, focusing on analysing the improvements in readability from the original texts to their simplified versions. In addition, we propose baseline methodologies for automatically generating the simplifications, translations and joint translation and simplifications contained in our dataset. Keywords: Text Simplification, Machine Translation, Dataset, COVID-19 1. Introduction The TICO-19 (Translation Initiative for COVID-19) dataset (Anastasopoulos et al., 2020) was developed in the wake of the COVID-19 pandemic as a bench- mark for Machine Translation (MT) systems. It con- tains 3,000 sentences about COVID-19, translated from English into 36 languages to allow researchers, indus- try stakeholders and policy makers to make informed decisions about the abilities of MT technology for pro- cessing this type of specialised content. The readability level of the texts in the TICO-19 dataset is generally high, with the English portion receiving a Flesch–Kincaid Grade Level (Kincaid et al., 1975) score of 12.14 for the development set and 12.06 for the test set.1 These values indicate that the texts could be understood by those finishing a high school education (grade 12). The Flesch-Szigriszt Index (Szigriszt Pa- zos, 2001) for Spanish readability indicates that the de- velopment and test sets in the Spanish portion of the corpus are at a similar level of difficulty to the English sets. This is counter-intuitive as the TICO-19 initiative is designed to promote the wide distribution of scien- tific literature across speakers of different languages. However, if the translated version of the medical liter- ature uses technical language that is inaccessible to a lay reader, then relevant information might not reach a wider audience in the general public. Consider the example in Table 1. While the Spanish translation is faithful to the original English sentence, it can be difficult to understand, and reflects the medical language of the original text. However, the simplified translation could be more suitable for lay readers. For instance, the word “subyacentes” (underlying) in the 1Scores computed …

Content truncated to 500 words