Identifying Broken Plurals in Unvowelised Arabic Tex

Identifying Broken Plurals in Unvowelised Arabic Text Abduelbaset Goweder University of Essex Dept. of Computer Science Wivenhoe Park, Colchester CO4 3SQ, UK agowed@essex.ac.uk Massimo Poesio University of Essex Dept. of Computer Science Wivenhoe Park, Colchester CO4 3SQ, UK poesio@essex.ac.uk Anne De Roeck The Open University Dept. of Computing Walton Hall, Milton Keynes Buckinghamshire, MK7 6AA, UK A.DeRoeck@open.ac.uk Jeff Reynolds University of Essex Dept. of Computer Science Wivenhoe Park, Colchester CO4 3SQ, UK reynt@essex.ac.uk Abstract Irregular (so-called broken) plural identification in modern standard Arabic is a problematic issue for information retrieval (IR) and language engineering applications, but their effect on the performance of IR has never been examined. Broken plurals (BPs) are formed by altering the singular (as in English: tooth � teeth) through an application of interdigitating patterns on stems, and singular words cannot be recovered by standard affix stripping stemming techniques. We developed several methods for BP detection, and evaluated them using an unseen test set. We incorporated the BP detection component into a new light-stemming algorithm that conflates both regular and broken plurals with their singular forms. We also evaluated the new light-stemming algorithm within the context of information retrieval, comparing its performance with other stemming algorithms. 1. Introduction Broken plurals constitute ~10% of texts in large Arabic corpora (Goweder and De Roeck, 2001), and ~41% of plurals (Boudelaa and Gaskell, 2002). Detecting broken plurals is therefore an important issue for light-stemming algorithms developed for applications such as information retrieval, yet the effect of broken plural identification on the performance of information retrieval systems has not been examined. We present several methods for BP detection, and evaluate them using an unseen test set containing 187,309 words. We also developed a new light-stemming algorithm incorporating a BP recognition component, and evaluated it within an information retrieval context, comparing its performance with other stemming algorithms. We give a brief overview of Arabic in Section 2. Several approaches to BP detection are discussed in Section 3, and their evaluation in Section 4. In Section 5, we present an improved light stemmer and its evaluation. Finally in Section 6, our conclusions are summarised. 2. Arabic Morphology and its Number System Arabic is a heavily inflected language. Its grammatical system is traditionally described in terms of a root-and-pattern structure, with about 10,000 roots (Ali, 1988). Roots such as drs (���) and ktb (���) are listed alphabetically in standard Arabic dictionaries like the Wehr-Cowan (Beesley, 1996). The root is the most basic verb form. Roots are categorized into: triliteral, quadriliteral, or rarely pentaliteral. Most words are derived from a finite set of roots formed by adding diacritics1 or affixes (prefixes, suffixes, and infixes) through an application of fixed patterns which are templates to help in deriving inflectional and derivational forms of a word. Theoretically, several hundreds of Arabic words can be derived from a single root. Traditional Arab grammarians describe Arabic morphology in terms of patterns associated with the basic root f3l (���, “to do”)- where f, 3, and l are like wildcards in …

Content truncated to 500 words