Clitics in Arabic Language: A Statistical Study

Clitics in Arabic Language: A Statistical Study * Fahad Alotaiby a, Salah Foda a, and Ibrahim Alkharashi b aDepartment of Electrical Engineering, King Saud University, P.O. Box 800, Riyadh 11421, Saudi Arabia {falotaiby, sfoda}@hotmail.com bComputer and Electrical Research Institute, King Abdulaziz City for Science and Technology, P.O. Box 6086, Riyadh 11442, Saudi Arabia kharashi@kacst.edu.sa Abstract. Clitics in Arabic language can be attached to a stem or to each other without orthographic marks such as an apostrophe. In this paper we present a statistical study of clitics and its effect in Arabic language. We tokenize large Arabic text using white-spaces and an automatic clitics tokenizer (AMIRA 2.0) and compare the unique-word count in both cases with English language. We also show the resulted distribution of clitics in Arabic and examine the performance of the used tokenizer. Using a 600 million words Arabic corpus, we report that the corresponding lexicon size could be reduced by 24.54% when applying clitics tokenization. Keywords: Arabic, clitics, statistics. * This work has been supported by a direct grant from His Excellency the Rector of King Saud University Prof. Abdullah Bin Abdulrahman Al-Othman, and by a grant from the Research Center, College of Engineering, King Saud University. Copyright 2010 by Fahad Alotaiby, Salah Foda, and Ibrahim Alkharashi 1 Introduction Arabic language is a rich and complex language. In Arabic, clitics are attached to a stem or to each other without any orthographic marks (i.e. an apostrophe). Number of clitics in Arabic is limited. When concatenated, clitics can generate a chain of up to four clitics before the stem (proclitics) and three clitics after the stem (enclitics). This may reduce the total number of white- spaces delimited words in a script but increases the size of the corresponding lexicon. Alotaiby et al. (2009) showed that the total number of unique words was about 2.2 million in a 600 million words Arabic corpus while in an equivalent English corpus it was 1.26 million. This implies the need of an Arabic lexicon of a size equivalent to 1.76 times the size of an English lexicon used to cover a similar broad linguistic content. A justification of this ratio was the heavy existence of clitics in Arabic language. A good approach to investigate this is by extracting clitics out of stems and performing comparative statistical analysis. 1.1 Arabic Language Arabic language is a Semitic language that is spoken by more than 280 million. Classical Arabic writing system was originally consonantal and written from right to left. Every letter in the 28 Arabic alphabets represents a single consonant. To overcome the problem of different pronunciations of consonants in Arabic text, graphical signs known as diacritics were invented in the seventh century. Currently in the Modern Standard Arabic (MSA), diacritics are omitted from written text almost all the time. As a result, this omission increases the number homographs PACLIC 24 Proceedings 595 (words with the same writing form). However, Arab readers normally differentiate between homographs by the context of the script. Moreover, Arabic …

Content truncated to 500 words