Automatic Diacritization for Low-Resource Languages Using a Hybrid Word and Consonant CMM

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 519–527, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Automatic Diacritization for Low-Resource Languages Using a Hybrid Word and Consonant CMM Robbie A. Haertel, Peter McClanahan, and Eric K. Ringger Department of Computer Science Brigham Young University Provo, Utah 84602, USA rah67@cs.byu.edu, petermcclanahan@gmail.com, ringger@cs.byu.edu Abstract We are interested in diacritizing Semitic lan- guages, especially Syriac, using only dia- critized texts. Previous methods have required the use of tools such as part-of-speech taggers, segmenters, morphological analyzers, and lin- guistic rules to produce state-of-the-art results. We present a low-resource, data-driven, and language-independent approach that uses a hybrid word- and consonant-level conditional Markov model. Our approach rivals the best previously published results in Arabic (15% WER with case endings), without the use of a morphological analyzer. In Syriac, we re- duce the WER over a strong baseline by 30% to achieve a WER of 10.5%. We also report results for Hebrew and English. 1 Introduction Abjad writing systems omit vowels and other di- acritics. The ability to restore these diacritics is useful for personal, industrial, and governmental purposes—especially for Semitic languages. In its own right, the ability to diacritize can aid language learning and is necessary for speech-based assis- tive technologies, including speech recognition and text-to-speech. Diacritics are also useful for tasks such as segmentation, morphological disambigua- tion, and machine translation, making diacritization important to Natural Language Processing (NLP) systems and intelligence gathering. In alphabetic writing systems, similar techniques have been used to restore accents from plain text (Yarowsky, 1999) and could be used to recover missing letters in the compressed writing styles found in email, text, and instant messages. We are particularly interested in diacritizing Syr- iac, a low-resource dialect of Aramaic, which pos- sesses properties similar to Arabic and Hebrew. This work employs conditional Markov models (CMMs) (Klein and Manning, 2002) to diacritize Semitic (and other) languages and requires only diacritized texts for training. Such an approach is useful for languages (like Syriac) in which annotated data and linguistic tools such as part-of-speech (POS) tag- gers, segmenters, and morphological analyzers are not available. Our main contributions are as follows: (1) we introduce a hybrid word and consonant CMM that allows access to the diacritized form of the pre- vious words; (2) we introduce new features avail- able in the proposed model; and (3) we describe an efficient, approximate decoder. Our models signifi- cantly outperform existing low-resource approaches across multiple related and unrelated languages and even achieve near state-of-the-art results when com- pared to resource-rich systems. In the next section, we review previous work rel- evant to our approach. Section 3 then motivates and describes the models and features used in our frame- work, including a description of the decoder. We describe our data in Section 4 and detail our exper- imental setup in Section 5. Section 6 presents our results. Finally, Section 7 briefly discusses our con- clusions and offers ideas for future …

Content truncated to 500 words