Highly Effective Arabic Diacritization using Sequence to Sequence Modeling

Proceedings of NAACL-HLT 2019, pages 2390–2395 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2390 Highly Effective Arabic Diacritization using Sequence to Sequence Modeling Hamdy Mubarak Ahmed Abdelali Hassan Sajjad Younes Samih Kareem Darwish Qatar Computing Research Institute, HBKU Research Complex, Doha 5825, Qatar {hmubarak,aabdelali,hsajjad,ysamih,kdarwish}@qf.org.qa Abstract Arabic text is typically written without short vowels (or diacritics). However, their pres- ence is required for properly verbalizing Ara- bic and is hence essential for applications such as text to speech. There are two types of dia- critics, namely core-word diacritics and case- endings. Most previous works on automatic Arabic diacritic recovery rely on a large num- ber of manually engineered features, particu- larly for case-endings. In this work, we present a unified character level sequence-to-sequence deep learning model that recovers both types of diacritics without the use of explicit fea- ture engineering. Specifically, we employ a standard neural machine translation setup on overlapping windows of words (broken down into characters), and then we use voting to se- lect the most likely diacritized form of a word. The proposed model outperforms all previ- ous state-of-the-art systems. Our best settings achieve a word error rate (WER) of 4.49% compared to the state-of-the-art of 12.25% on a standard dataset. 1 Introduction Arabic uses two types of vowels, namely long vowels, which are explicitly placed in the text, and short vowels, which are diacritic marks that are typically omitted during writing. In order to read Arabic words properly, readers need to rein- troduce the missing diacritics. Therefore, accu- rate diacritic recovery is essential for some ap- plications such as text-to-speech. There are, in turn, two types of Arabic diacritics, namely core- word diacritics (CW), which specify lexical se- lection, and case endings (CE), which typically indicate syntactic role. For example, the word “AlElm”1 can accept many possible core-word diacritics depending on the intended meaning 1In this paper, we use Buckwalter transliteration. such as: “AaloEalam” (the flag) and “AaloEilom” (the knowledge/science). In our training corpus, 17.1% of the word-cores have more than one valid diacritized form. In the sentence “Zahara AaloEalamu” (the flag appeared), “AaloEalamu” is the subject and takes the case ending “u”, and in the sentence “ra>ayotu AaloEalama” (I saw the flag), “AaloEalama” is the object and takes the case ending “a”. Aside from function words, past tense and accusative verb forms, and for- eign names, most words can accept different case- endings depending on context. In this paper, we introduce a unified model for both diacritic types while improving upon the state-of-the-art. Specifically, we approached the task as a sequence-to-sequence (seq2seq) prob- lem (Cho et al., 2014); taking advantage of the re- cent advancements in Neural Machine Translation (NMT) (Britz et al., 2017; Kuchaiev et al., 2018) among other applications where seq2seq models made a breakthrough (Yu et al., 2016; Witten et al., 2016; Abadi et al., 2016). Using the analogy of translation which employs a sequential encoder and a sequential decoder, the input undiacritized text will be encoded …

Content truncated to 500 words