Improving Arabic Diacritization through Syntactic Analysis

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1309–1315, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Improving Arabic Diacritization through Syntactic Analysis Anas Shahrour, Salam Khalifa and Nizar Habash Computational Approaches to Modeling Language Lab New York University Abu Dhabi United Arab Emirates {anas.shahrour,salamkhalifa,nizar.habash}@nyu.edu Abstract We present an approach to Arabic auto- matic diacritization that integrates syntac- tic analysis with morphological tagging through improving the prediction of case and state features. Our best system in- creases the accuracy of word diacritization by 2.5% absolute on all words, and 5.2% absolute on nominals over a state-of-the- art baseline. Similar increases are shown on the full morphological analysis choice. 1 Introduction Modern Standard Arabic (MSA) orthography generally omits diacritical marks which encode lexical as well as syntactic (case) information. The task of Arabic automatic diacritization is about the automatic restoration of the missing diacritics. Diacritization improvement in Arabic has impor- tant implications for downstream processing for Arabic natural language processing, e.g. speech recognition (Ananthakrishnan et al., 2005; Biadsy et al., 2009), speech synthesis (Elshafei et al., 2002), and machine translation (Diab et al., 2007; Zbib et al., 2010). Previous efforts on diacritization utilized mor- phological tagging techniques to disambiguate word forms. Habash et al. (2007a) observe that while such techniques work relatively well on lex- ical diacritics (located on word stems), they are much worse for syntactic case diacritics (typically word final). They suggest that syntactic anal- ysis may help with automatic diacrtization, but stop short of testing the idea, and instead demon- strate that complex linguistic features and rules are needed to model complex Arabic case using gold syntactic analyses. In this paper, we develop an approach for improving the quality of automatic Arabic diacritization through the use of automatic syntactic analysis. Our approach combines hand- written rules for case assignment and agreement with machine learning of case and state adjustment on the output of a state-of-the-art morphological tagger. Our best system increases the accuracy of word diacrtization by 2.5% absolute overall, and 5.2% absolute on nominals over a state-of-the-art baseline. 2 Linguistic Background Arabic automatic processing, and specifically dia- critization is hard for a number of reasons. First, Arabic words are morphologically rich. The morphological analyzer we use represents Arabic words with 15 features (Pasha et al., 2014).1 We focus on case and state in this paper. In our data set, case has five values: nominative (n), accusative (a), genitive (g), undefined (u) and not applicable (na). Cases n, a and g are usually expressed with an overt morpheme. Case u is used to mark words without an overt morpheme expres- sion of case (e.g., invariable nouns such as � ��� ��� šakwaý ‘complaint’), or those not assigned a case in the manual annotations. Most of the missing assignments are for foreign proper nouns, which often do not receive case markers. However, this is not done consistently in the training data we use. Case na is used for non-nominals. State is a …

Content truncated to 500 words