Induction of Root and Pattern Lexicon for Unsupervised Morphological Analysis of Arabic

International Joint Conference on Natural Language Processing, pages 1012–1016, Nagoya, Japan, 14-18 October 2013. Induction of Root and Pattern Lexicon for Unsupervised Morphological Analysis of Arabic Bilal Khaliq Dept of Informatics, University of Sussex Brighton BN1 9QJ, UK bk54@sussex.ac.uk John Carroll Dept of Informatics, University of Sussex Brighton BN1 9QJ, UK J.A.Carroll@sussex.ac.uk Abstract We propose an unsupervised approach to learning non-concatenative morphology, which we apply to induce a lexicon of Arabic roots and pattern templates. The approach is based on the idea that roots and patterns may be revealed through mutually recursive scoring based on hypothesized pattern and root frequencies. After a further iterative refinement stage, morphological analysis with the induced lexicon achieves a root identification accuracy of over 94%. Our approach differs from previous work on unsupervised learning of Arabic morphology in that it is applicable to naturally-written, unvowelled text. 1 Introduction Manual development of morphological analysis systems is expensive. It is impractical to develop morphological descriptions for more than a very small proportion of human languages. In recent years a number of approaches have been proposed that learn the morphology of a language from unannotated text. The Morpho Challenge and similar competitions have further motivated researchers to devise techniques for unsupervised learning of language. Previous work in unsupervised morphology learning has mostly addressed concatenative morphology, in which surface word forms are sequentially separated or segmented into morpheme units. However, some languages (in particular Semitic languages) have another type of word formation in which morphemes combine in a non-concatenative manner, through the interdigitation of a root morpheme with an affix or pattern template. Unsupervised learning of non-concatenative morphology has received comparatively little attention. In this paper we describe a conceptually simple yet effective unsupervised approach to learning non-concatenative morphology. We apply our approach to inducing an Arabic lexicon of trilateral roots and pattern templates. Lexicon acquisition is based on the idea that roots and affix patterns may be revealed by their converses, i.e. roots are identified from occurrences of patterns and conversely patterns are recognized from root frequencies. Subsequently, the lexicons are iteratively improved by refining the morpheme strengths computed in the previous step. The paper is organized as follows. We survey previous related work (Section 2), and then give a brief introduction to Arabic root and pattern morphology (Section 3). We explain our basic technique for unsupervised lexicon induction in Section 4, followed by the refinement procedure (Section 5). Section 6 describes how the lexicon is used for morphological analysis. Finally, we present an evaluation (Section 7) and conclusions (Section 8). 2 Related Work Beesley (1996) describes one of the first morphological analysis systems for Arabic, based on finite-state techniques with manually acquired lexicons and rules. This kind of approach, although potentially producing an efficient and accurate system, is expensive in time and linguistic expertise, and lacks robustness in terms of extendibility to word types not in the dictionary (Ahmed, 2000). Darwish (2002) describes a semi-automatic technique that learns morphemes and induces rules for deriving stems using an …

Content truncated to 500 words