Mean Machine Translations: On Gender Bias in Icelandic Machine Translations

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 3113–3121 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 3113 Mean Machine Translations: On Gender Bias in Icelandic Machine Translations Agnes Sólmundsdóttir, Dagbjört Guðmundsdóttir Lilja Björk Stefánsdóttir, Anton Karl Ingason University of Iceland Sæmundargata 2, 102 Reykjavík, Iceland {ags46, dagu, lbs, antoni}@hi.is Abstract This paper examines machine bias in language technology. Machine bias can affect machine learning algorithms when language models trained on large corpora include biased human decisions or reflect historical or social inequities, e.g. regarding gender and race. The focus of the paper is on gender bias in machine translation and we discuss a study conducted on Icelandic translations in the translation systems Google Translate and Vélþýðing.is. The results show a pattern which corresponds to certain societal ideas about gender. For example it seems to depend on the meaning of adjectives referring to people whether they appear in the masculine or feminine form. Adjectives describing positive personality traits were more likely to appear in masculine gender whereas the negative ones frequently appear in feminine gender. However, the opposite applied to appearance related adjectives. These findings unequivocally demonstrate the importance of being vigilant towards technology so as not to maintain societal inequalities and outdated views — especially in today’s digital world. Keywords: language technology, machine bias, gender bias, machine translation, gender 1. Introduction The main purpose of this paper is to examine how gender bias appears in English-Icelandic transla- tions. A study was conducted on Icelandic transla- tions in the translation systems Google Translate and Vélþýðing.is.1 It was conducted over a period of 20 months where Google Translate was tested three times providing data on how the representations of gender bias can fluctuate over time. Vélþýðing.is, a transla- tion tool focusing on Icelandic, was tested alongside the multilingual Google Translate in the latest period to have a comparison between two different translation tools in order to see if a similar gender bias occurs. Our main research question is: How does gender-bias appear in English-Icelandic translations and how does it vary over a 20 month period as the systems evolve? The paper is structured as follows. Section 2 gives an overview of previous research concerning machine bias, focusing on gender bias in machine translation tools. In Section 3, we describe the methodology of the study and Section 4 gives statistical information on how gender bias is manifested in both translation tools, comparing the results. Finally, we conclude with Sec- tion 5. 2. Background In the field of Machine Translations (MT), statistical and neural network approaches have been dominant in recent years (Zong and Hong, 2018). Statistical and 1The following abbreviations and acronyms are used in the paper: FEM=feminine, MASC=masculine, NLP=Natural Language Processing, STEM=Science, Technology, Engi- neering and Mathematics. neural machine translation systems are data-driven in the sense that large collections of linguistic data are used to train these models for natural language pro- cessing (NLP). Such methods rely heavily on paral- lel …

Content truncated to 500 words