A Large-Scale Leveled Readability Lexicon for Standard Arabic

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3053–3062 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3053 A Large-Scale Leveled Readability Lexicon for Standard Arabic Muhamed Al Khalil, Nizar Habash, Zhengyang Jiang Computational Approaches to Modeling Language (CAMeL) Lab New York University Abu Dhabi {muhamed.alkhalil, nizar.habash, zj522}@nyu.edu Abstract We present a large-scale 26,000-lemma leveled readability lexicon for Modern Standard Arabic. The lexicon was manually annotated in triplicate by language professionals from three regions in the Arab world. The annotations show a high degree of agreement; and major differences were limited to regional variations. Comparing lemma readability levels with their frequencies provided good insights in the benefits and pitfalls of frequency-based readability approaches. The lexicon will be publicly available. Keywords: Readability, Lexicon, Arabic 1. Introduction Modeling readability levels is relevant to a range of natural language processing (NLP) tasks from developing language education applications to user profiling. Much work has been done on readability leveling and its assessment and specification in English leading to the development of many resources and tools. However, this is not the case for many other languages. The work presented in this paper is part of a project on Sim- plification of Arabic Masterpieces for Extensive Reading (SAMER) (Al Khalil et al., 2017; Al Khalil et al., 2018). Specifically, we discuss the challenges of, and solutions to, the development of a large-scale leveled readability lexi- con for Modern Standard Arabic (MSA). Although some aspects of the effort are Arabic specific, we situate it within the larger frameworks of approaches to readability lexicon development. Our contributions include the following: • We define a five-level readability scale for MSA tar- geting native speakers and create annotation guide- lines for it. • We manually annotate a 26,578-lemma lexicon in trip- licate by language professionals from three different regions in the Arab world. • We report on a detailed analysis of the major disagree- ments among our annotators. • We carefully study the relationship between frequency-based and intuition-based readability classifications using our newly created lexicon. • We make this new lexicon publicly available to sup- port research and tool development for Arabic read- ability tasks.1 The rest of this paper is structured as follows. We start with three background sections on approaches to readabil- ity lexicon development (Section 2), previous related work (Section 3), and relevant linguistic facts about Arabic (Sec- tion 4). Section 5 details our approach including guideline 1http://resources.camel-lab.com/ creation, data extraction, and manual annotation. We dis- cuss our results in Section 6. Section 7 concludes and maps some future work directions. 2. Approaches to Readability Lexicons We present next two different approaches to the develop- ment of leveled readability lexicons: the frequency ap- proach and the intuitive approach. 2.1. The Frequency Approach Frequency and frequency-derived measures have allowed for the study of language usage on a massive scale, and produced many innovative advancements in the mapping and understanding of language, especially for teaching and learning purposes. Examples …

Content truncated to 500 words