Finite State Machine Pattern-Root Arabic Morphological Generator, Analyzer and Diacritizer

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3834–3841 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3834 Finite State Machine Pattern-Root Arabic Morphological Generator, Analyzer and Diacritizer Maha Alkhairy*, Afshan Jafri, David A. Smith* *College of Computer and Information Science Northeastern University, College of Computer and Information Science King Saud University alkhairy.m@husky.neu.edu, ajafri@ksu.edu.sa, dasmith@ccs.neu.edu Abstract We describe and evaluate the Finite-State Arabic Morphologizer (FSAM) – a concatenative (prefix-stem-suffix) and templatic (root- pattern) morphologizer that generates and analyzes undiacritized Modern Standard Arabic (MSA) words, and diacritizes them. Our bidirectional unified-architecture finite state machine (FSM) is based on morphotactic MSA grammatical rules. The FSM models the root-pattern structure related to semantics and syntax, making it readily scalable unlike stem-tabulations in prevailing systems. We evaluate the coverage and accuracy of our model, with coverage being percentage of words in Tashkeela (a large corpus) that can be analyzed. Accuracy is computed against a gold standard, comprising words and properties, created from the intersection of UD PADT treebank and Tashkeela. Coverage of analysis (extraction of root and properties from word) is 82%. Accuracy results are: root computed from a word (92%), word generation from a root (100%), non-root properties of a word (97%), and diacritization (84%). FSAM’s non-root results match or surpass MADAMIRA’s, and root result comparisons are not made because of the concatenative nature of publicly available morphologizers. Keywords: Morphological Analysis, Concatenative Morphology, Templatic Morphology, Root-Pattern Analysis, Morphological Generation, Diacritization, Finite State Machine 1. Introduction Morphological analyzers support part-of-speech tagging, syntactic parsing, and semantic processing in computa- tional linguistics and have applications in natural language processing from information retrieval to speech synthesis to machine translation. Arabic utilizes form-based morphology, which considers the form of units making up a word, their interactions with each other and how they relate to the word’s over- all form, and has concatenative and templatic morphemes (Farghaly, 2010; Habash, 2010). Concatenative morphol- ogy is centered on stem and affix (prefixes, suffixes, cir- cumfixes) morphemes, which are generally concatenated in a sequence to produce a surface form. A morphological grammar that constructs stems from interdigitation (inter- leaving) of the root and pattern is called templative mor- phology. The root provides the core meaning of a word whereas the pattern provides the properties (gender, num- ber, etc.), category (regular/irregular verb and noun, proper noun, function word), and syntactic position. Figure 1 illustrates the structure of Arabic words using the word fasmiEahaA1 where the prefix is fa (so), the suffix is haA (her), and the stem is samiEa (he heard) with the root of s m E , which gives the core meaning of hearing, and pattern faEila which gives the properties of regular verb, past, singular, masculine to the word giving the final meaning of “so he heard her”. Our model (FSAM) is an acceptor (recognizer), synthesizer (generator), and analyzer (parser) of Modern Standard Ara- bic (MSA) words. It is also a diacritizer that infers short vowels and other diacritics from …

Content truncated to 500 words