CrossAligner & Co: Zero-Shot Transfer Methods for Task-Oriented Cross-lingual Natural Language Understanding

Findings of the Association for Computational Linguistics: ACL 2022, pages 364 - 371 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Towards Responsible Natural Language Annotation for the Varieties of Arabic A. Stevie Bergman Responsible AI, Meta / New York, NY asbergman@fb.com Mona T. Diab Responsible AI, Meta / Seattle, WA mdiab@fb.com Abstract When building NLP models, there is a tendency to aim for broader coverage, often overlooking cultural and (socio)linguistic nuance. In this position paper, we make the case for care and at- tention to such nuances, particularly in dataset annotation, as well as the inclusion of cultural and linguistic expertise in the process. We present a playbook for responsible dataset cre- ation for polyglossic, multidialectal languages. This work is informed by a study on Arabic annotation of social media content. 1 Introduction Natural language processing (NLP) is the foun- dation of numerous automated decision-making systems in a growing number of scenarios and languages, including content moderation on plat- forms with global reach and consequence (Gille- spie, 2020). Thus it is highly pertinent to address how practitioners can build responsible NLP sys- tems, models, and workflows for languages beyond English (Bender, 2019; Mielke, 2016; Husain and Uzuner, 2021). In doing so, it is essential that these systems are designed with the inclusion of domain experts and stakeholder groups with na- tive fluency and local/regional knowledge (Bender, 2009; Ovadya and Whittlestone, 2019). This will not only ensure the presence of the deep problem understanding necessary to create accurate systems and anticipate potential harms, but foster earned trust and legitimacy in the system (Martin Jr et al., 2020). Many performant machine learning/NLP algo- rithms to date are supervised, relying on large scale annotated training data, thus the veracity and cura- tion of the data labels can have significant impact on model performance (Bender et al., 2021; North- cutt et al., 2021). And further, even unsupervised and semi-supervised systems require labeled data for evaluation as a bare minimum to allow visibility into any blindspots in the system performance. In this position paper, we focus on NLP datasets, highlighting the potential for compounding harms to at-risk populations and calling for greater care and attention to annotation and annotator support (Denton et al., 2021). This is demonstrated via the varieties of Arabic.1 2 Arabic Varieties and NLP Arabic is a Semitic language, spoken by over 420M people globally with the highest concentration in the Middle East and North Africa (MENA) where it is the dominant or official language of over twenty nations. Arabic is better described as a family of languages as the so-called "dialects" are highly vari- able and often have low inter-dialect comprehensi- bility (Al-Wer and Horesh, 2018). e.g. Moroccan and Egyptian varieties are about as mutually in- telligible as Spanish and Romanian. Importantly, Arabic should not be considered as an analog to English, i.e. one language with closely related dialects (e.g. British, American, and Australian English). This comparison buries the polyglossic (when languages co-exist in a community) and het- erogeneous characteristics …

Content truncated to 500 words