A Compression Technique for Arabic Dictionaries: The Affix Analysis

A COMPRESSION TECHNIQI~ FOR ARABIC DICTIONARIES : THE AFFIX ANALYSIS. Abdelmajid BEN HAMADOU D~partement of computer science -FSEG Faculty B.P 69 - Route de l'a~roport - SFAX - TUNISIA ABSTRACT In every application that concerns the automatic processing of natural language, the problem of the dictionary size is posed. In this paper , we propose a compression dictionary al~orithm based on an affix ana- lysis of the non diacritical Arabic. It consists in decomposing a word into its first elements taking into account the different linguistic transformations that can affect the morphological structures. This work has been achieved as part of a study of the automatic detection and correction of spelling errors in the non diacritical Arabic texts. I- INTRODUCTION In every application that concerns the automatic processing of natural language, the problem of the dictionary size is posed. We can approach this impor- tant question in several ways and particularly : - By grouping together the common prefixes of the different language words. In the PIAF system,(interac- tive program for French Analysis) for instance, words are represented in chained lists following an alpha- betical order [COUR 77] EX : PARTIEL ~ PARTIES_____--~_PARTOUT ... PARTIE-~ J PART - , ~ - - - By creating multiple dictionaries: or:efor each major topic area. This approach requires, in addition, a common base dictionary. When a particular area is concerned, a temporary master dictionary is created by increasinF the base dictionary with selected local ones. -- By usin~ the Affix analysis which consists in performing a morphological analysis in order to iden- tify, in a given word, the redundant elements (Affixes). The dictionary will be limited to the non- redundant elements (roots). This technique is used specially in the DECIO - SPELL system for detecting and correcting spelling errors. In the present paper, we will develop this last approach for the non_diacritical Arabic. The particularities of the algorithms that we propose, stem~ in great part, form the specificities of the language used : - Words are written in consonantic form - Words can contain infixes - Morphological structures can be altered by linguistic transformations. This work has been developed within a national research project for the study of the automatic detec- tion and correction of spelling errors in Arabic texts BEN 86 7 II - THEORETICAL ASPECTS Let V be a finite Set and V ~, the set of words built on V including nul s$ing noted W ~-V-- W = W I W 2 ...... W . W i ~ V n i 6 fl ,mr let v+ = v'- 1~} i°/ Prefix ( W ) let W = W 1 W 2 ....... W n. W 6 V + We call order i prefix the quantity Pi = W] W2.-W i (l g i <n-l) the order o prefix is 2°/ Suffix ( W ) + Let W = W 1 W 2 ..... W W ~ V n We call order j suffix the quantity S. = W. Wj+I ........... …

Content truncated to 500 words