Arabic Diacritization with Recurrent Neural Networks

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2281–2285, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Arabic Diacritization with Recurrent Neural Networks Yonatan Belinkov and James Glass Massachusetts Institute of Technology Computer Science and Artificial Intelligence Laboratory {belinkov, glass}@mit.edu Abstract Arabic, Hebrew, and similar languages are typi- cally written without diacritics, leading to ambigu- ity and posing a major challenge for core language processing tasks like speech recognition. Previous approaches to automatic diacritization employed a variety of machine learning techniques. However, they typically rely on existing tools like morpho- logical analyzers and therefore cannot be easily extended to new genres and languages. We de- velop a recurrent neural network with long short- term memory layers for predicting diacritics in Arabic text. Our language-independent approach is trained solely from diacritized text without re- lying on external tools. We show experimentally that our model can rival state-of-the-art methods that have access to additional resources. 1 Introduction Hebrew, Arabic, and other languages based on the Arabic script usually represent only consonants in writing and do not mark vowels. In such writ- ing systems, diacritics are used for marking short vowels, gemination, and other phonetic units. In practice, diacritics are usually restricted to specific settings such as language teaching or to religious texts. Faced with a non-diacritized word, readers infer missing diacritics based on their prior knowl- edge and the context of the word in order to re- solve ambiguities. For example, Maamouri et al. (2006) mention several types of ambiguity for the Arabic string �� � Elm, both within and across part-of-speech tags, and at a grammatical Word Gloss Ealima he knew Eulima it was known Eal~ama he taught Eilomu knowledge (def.nom) ... ... EilomK knowledge (indef.gen) Ealamu flag (def.nom) ... ... EalamK flag (indef.gen) Table 1: Possible diacritized forms for ��� Elm. level. In practice, a morphological analyzer like MADA (Habash et al., 2009) produces at least 13 different diacritized forms for this word, a subset of which is shown in Table 1.1 The ambiguity in Arabic orthography presents a problem for many language processing tasks, in- cluding acoustic modeling for speech recognition, language modeling, text-to-speech, and morpho- logical analysis. Automatic methods for diacriti- zation aim to restore diacritics in a non-diacritized text. While earlier work used rule-based meth- ods, more recent studies attempted to learn a di- acritization model from diacritized text. A vari- ety of methods have been used, including hidden Markov models, finite-state transducers, and max- imum entropy – see the review in (Zitouni and Sarikaya, 2009) – and more recently, deep neu- ral networks (Al Sallab et al., 2014). In addi- tion to learning from diacritized text, these meth- ods typically rely on external resources such as part-of-speech taggers and morphological analyz- ers like the MADA tool (Habash and Rambow, 2007). However, building such resources is a labor-intensive task and cannot be easily extended to new languages, dialects, and domains. 1Arabic transliteration follows the Buckwalter scheme: http://www.qamus.org/transliteration.htm. 2281 Diacritic Transliteration …

Content truncated to 500 words