SimplifyUR: Unsupervised Lexical Text Simplification for Urdu

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3484–3489 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3484 SimplifyUR: Unsupervised Lexical Text Simplification for Urdu Namoos Hayat Qasmi1, Haris Bin Zia1, Awais Athar2, Agha Ali Raza1 1Information Technology University, 6th Floor, Arfa Software Technology Park, Ferozepur Road, Lahore, Pakistan 2European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Cambridge, CB10 1SD, UK {namoos.qasmi, haris.zia, agha.ali.raza}@itu.edu.pk awais@ebi.ac.uk Abstract This paper presents the first attempt at Automatic Text Simplification (ATS) for Urdu, the language of 170 million people worldwide. Being a low-resource language in terms of standard linguistic resources, recent text simplification approaches that rely on manually crafted simplified corpora or lexicons such as WordNet are not applicable to Urdu. Urdu is a morphologically rich language that requires unique considerations such as proper handling of inflectional case and honorifics. We present an unsupervised method for lexical simplification of complex Urdu text. Our method only requires plain Urdu text and makes use of word embeddings together with a set of morphological features to generate simplifications. Our system achieves a BLEU score of 80.15 and SARI score of 42.02 upon automatic evaluation on manually crafted simplified corpora. We also report results for human evaluations for correctness, grammaticality, meaning-preservation and simplicity of the output. Our code and corpus are publicly available to make our results reproducible. Keywords: automatic text simplification, lexical simplification, Urdu 1. Introduction Text simplification has gathered much attention recently because of its interesting applications in machine translation (Mishra et al., 2014; Štajner and Popovic, 2016) and language learning for people with learning disabilities (Carroll et al., 1998; Chen et al., 2016). Lexically complex text is also difficult to understand for children and non- native speakers (Petersen and Ostendorf, 2007; De Belder and Moens, 2010). Automatic Text Simplification (ATS) to replace lexically complex words with their simpler equivalents is thus necessary to make text understandable for a wide variety of audiences. Urdu, the official language of Pakistan, is an Indo-Aryan language with over 170 million speakers throughout Pakistan, India and several other countries1. It uses Arabic script which is written from right to left. Text Simplification for Urdu exhibits a number of challenges due to its morphological richness, use of case markers and lack of linguistic resources. Urdu is a low resource language in terms of standard linguistic resources (Cieri et al., 2016). Therefore, neither data-driven approaches such as Neural Text Simplification (NTS) (Nisioi et al., 2017) that rely on complex-simplified parallel corpora, nor rule- based methods that depend on lexical resources such as WordNet2 for generating simplifications are applicable. Glavaš and Štajner (2015) proposed LIGHT-LS, a resource-light lexical simplification method that works well for languages with moderate morphological diversity such as English. Our approach adapts LIGHT-LS to handle challenges of Urdu text simplification by using additional features and achieves significant improvement in performance (18.4%) over the LIGHT-LS baseline. We also present a new manually constructed parallel corpus of complex-simplified Urdu sentences for benchmarking …

Content truncated to 500 words