A Multitask Learning Approach for Diacritic Restoration

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8238–8247 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 8238 A Multitask Learning Approach for Diacritic Restoration Sawsan Alqahtani 1,2 and Ajay Mishra1 and Mona Diab2∗ 1AWS, Amazon AI 2The George Washington University sawsa@amazon.com, misaja@amazon.com, mtdiab@gwu.edu Abstract In many languages like Arabic, diacritics are used to specify pronunciations as well as mean- ings. Such diacritics are often omitted in written text, increasing the number of possi- ble pronunciations and meanings for a word. This results in a more ambiguous text making computational processing on such text more difficult. Diacritic restoration is the task of restoring missing diacritics in the written text. Most state-of-the-art diacritic restoration mod- els are built on character level information which helps generalize the model to unseen data, but presumably lose useful information at the word level. Thus, to compensate for this loss, we investigate the use of multi-task learning to jointly optimize diacritic restora- tion with related NLP problems namely word segmentation, part-of-speech tagging, and syn- tactic diacritization. We use Arabic as a case study since it has sufficient data resources for tasks that we consider in our joint modeling. Our joint models significantly outperform the baselines and are comparable to the state-of- the-art models that are more complex relying on morphological analyzers and/or a lot more data (e.g. dialectal data). 1 Introduction In contrast to English, some vowels in languages such as Arabic and Hebrew are not part of the alpha- bet and diacritics are used for vowel specification.1 In addition to pertaining vowels, diacritics can also represent other features such as case marking and phonological gemination in Arabic. Not including diacritics in the written text in such languages in- creases the number of possible meanings as well as pronunciations. Humans rely on the surrounding ∗*The work was conducted while the author was with AWS, Amazon AI. 1Diacritics are marks that are added above, below, or in- between the letters to compose a new letter or characterize the letter with a different sound (Wells, 2000). context and their previous knowledge to infer the meanings and/or pronunciations of words. How- ever, computational models, on the other hand, are inherently limited to deal with missing diacritics which pose a challenge for such models due to increased ambiguity. Diacritic restoration (or diacritization) is the pro- cess of restoring these missing diacritics for ev- ery character in the written texts. It can spec- ify pronunciation and can be viewed as a relaxed variant of word sense disambiguation. For exam- ple, the Arabic word ��� Elm2 can mean “flag” or “knowledge”, but the meaning as well as pronun- ciation is specified when the word is diacritized ( ������ Ealamu means “flag” while ������ Eilomo means “knowledge”). As an illustrative example in En- glish, if we omit the vowels in the word pn, the word can be read as pan, pin, pun, and pen, each of these variants have different pronunciations and meanings if it composes …

Content truncated to 500 words