Joint Diacritization, Lemmatization, Normalization, and Fine-Grained Morphological Tagging

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8297–8307 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 8297 Joint Diacritization, Lemmatization, Normalization, and Fine-Grained Morphological Tagging Nasser Zalmout and Nizar Habash Computational Approaches to Modeling Language Lab New York University Abu Dhabi United Arab Emirates {nasser.zalmout,nizar.habash}@nyu.edu Abstract The written forms of Semitic languages are both highly ambiguous and morphologically rich: a word can have multiple interpretations and is one of many inflected forms of the same concept or lemma. This is further ex- acerbated for dialectal content, which is more prone to noise and lacks a standard orthogra- phy. The morphological features can be lexi- calized, like lemmas and diacritized forms, or non-lexicalized, like gender, number, and part- of-speech tags, among others. Joint modeling of the lexicalized and non-lexicalized features can identify more intricate morphological pat- terns, which provide better context modeling, and further disambiguate ambiguous lexical choices. However, the different modeling gran- ularity can make joint modeling more difficult. Our approach models the different features jointly, whether lexicalized (on the character- level), or non-lexicalized (on the word-level). We use Arabic as a test case, and achieve state- of-the-art results for Modern Standard Arabic with 20% relative error reduction, and Egyp- tian Arabic with 11% relative error reduction. 1 Introduction Morphological modeling in Semitic languages is challenging. Their optional short vowels (diacrit- ics) increase the overall ambiguity of surface forms; and their morphological richness results in large target spaces, which increase model sparsity. The different morphological features can be modeled through combined feature tags, using a single (but very large) target space, or through having separate models for each of the features. The combined fea- tures approach models the relationships between the different features explicitly, but the large tar- get spaces for morphologically rich languages fur- ther increase sparsity. On the other hand, separate feature modeling guarantees smaller target spaces for the individual features, but the hard separation between the features prevents modeling any inter- feature dependencies. The set of morphological features includes lexicalized and non-lexicalized features, which further exacerbates joint modeling. Non-lexicalized features, like gender, and number, among others, have limited target spaces, and usu- ally modeled as tagging tasks. Lexicalized features, like lemmas and diacritized forms (for Semitic lan- guages), are open-ended, with large target vocabu- laries. Moreover, non-lexicalized features are mod- eled on the word level, whereas lexicalized features are optimally modeled on the character level. This difference in the modeling granularity can be chal- lenging for joint models. In this paper we present a model for handling lexicalized and non-lexicalized features jointly. We use a sequence-to-sequence architecture, with dif- ferent parameter sharing strategies at the encoder and decoder sides for the different features. The non-lexicalized features are handled with a tagger, which shares several parameters with the encoder, and uses a multitask-learning architecture to model the different non-lexicalized features jointly. The lexicalized features, on the other hand, are handled with a specific decoder for each feature, sharing the same …

Content truncated to 500 words