Adversarial Multitask Learning for Joint Multi-Feature and Multi-Dialect Morphological Modeling

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1775–1786 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 1775 Adversarial Multitask Learning for Joint Multi-Feature and Multi-Dialect Morphological Modeling Nasser Zalmout and Nizar Habash Computational Approaches to Modeling Language Lab New York University Abu Dhabi United Arab Emirates {nasser.zalmout,nizar.habash}@nyu.edu Abstract Morphological tagging is challenging for mor- phologically rich languages due to the large target space and the need for more train- ing data to minimize model sparsity. Di- alectal variants of morphologically rich lan- guages suffer more as they tend to be more noisy and have less resources. In this pa- per we explore the use of multitask learning and adversarial training to address morpho- logical richness and dialectal variations in the context of full morphological tagging. We use multitask learning for joint morphological modeling for the features within two dialects, and as a knowledge-transfer scheme for cross- dialectal modeling. We use adversarial train- ing to learn dialect invariant features that can help the knowledge-transfer scheme from the high to low-resource variants. We work with two dialectal variants: Modern Standard Ara- bic (high-resource “dialect”1) and Egyptian Arabic (low-resource dialect) as a case study. Our models achieve state-of-the-art results for both. Furthermore, adversarial training pro- vides more significant improvement when us- ing smaller training datasets in particular. 1 Introduction Morphological tagging for morphologically rich languages (MRL) involves modeling interdepen- dent features, with a large combined target space. Joint modeling of the different features, through feature concatenation, results in a large target space with increased sparsity. Whereas total sep- aration of the different feature models eliminates access to the other features, which constrains the model. These issues are further exacerbated for di- alectal content, with many morphosyntactic varia- tions that further complicate the modeling. 1We view Arabic as a collective of dialectal variants in which MSA is the main high-resource dialect, and EGY is a low-resource dialect. We therefore use “variant” and “di- alect” interchangeably. In this paper we work with Modern Standard Arabic (MSA) and Egyptian Arabic (EGY), both MRLs, and dialectal variants. Written Arabic text is also highly ambiguous, due to its diacritic- optional orthography, resulting in several interpre- tations of the same surface forms, and further in- creasing sparsity. Joint modeling is particularly promising for such ambiguous nature as it sup- ports identifying more complex patterns involving multiple features. In EGY, for example, the suf- fix ��� nA ‘we, us, our’ in the word ������ drsnA can be the subject of the perfective 1st person plural verb (‘we studied’), the 1st person plural object clitic of a perfective 3rd person masculine singu- lar verb (‘he taught us’), or the 1st person plural possessive pronoun for the nominal (‘our lesson’), among other possible interpretations. Morphological tagging models rely heavily on the availability of large annotated training datasets. Unlike MSA, Arabic Dialects are gen- erally low on resources. In this paper we also experiment with knowledge-transfer models from high to low-resource variants. …

Content truncated to 500 words