Breeding Gender-aware Direct Speech Translation Systems

Proceedings of the 28th International Conference on Computational Linguistics, pages 3951–3964 Barcelona, Spain (Online), December 8-13, 2020 3951 Breeding Gender-aware Direct Speech Translation Systems Marco Gaido1,2 †, Beatrice Savoldi2 †, Luisa Bentivogli1, Matteo Negri1, Marco Turchi1 1Fondazione Bruno Kessler, Trento, Italy 2University of Trento, Italy {mgaido,bentivo,negri,turchi}@fbk.eu,beatrice.savoldi@unitn.it Abstract In automatic speech translation (ST), traditional cascade approaches involving separate transcrip- tion and translation steps are giving ground to increasingly competitive and more robust direct solutions. In particular, by translating speech audio data without intermediate transcription, di- rect ST models are able to leverage and preserve essential information present in the input (e.g. speaker’s vocal characteristics) that is otherwise lost in the cascade framework. Although such ability proved to be useful for gender translation, direct ST is nonetheless affected by gender bias just like its cascade counterpart, as well as machine translation and numerous other natural language processing applications. Moreover, direct ST systems that exclusively rely on vocal biometric features as a gender cue can be unsuitable and potentially harmful for certain users. Going beyond speech signals, in this paper we compare different approaches to inform direct ST models about the speaker’s gender and test their ability to handle gender translation from English into Italian and French. To this aim, we manually annotated large datasets with speakers’ gen- der information and used them for experiments reflecting different possible real-world scenarios. Our results show that gender-aware direct ST solutions can significantly outperform strong – but gender-unaware – direct ST models. In particular, the translation of gender-marked words can increase up to 30 points in accuracy while preserving overall translation quality. 1 Introduction Language use is intrinsically social and situated as it varies across groups and even individuals (Bamman et al., 2014). As a result, the language data that are collected to build the corpora on which natural language processing models are trained are often far from being homogeneous and rarely offer a fair representation of different demographic groups and their linguistic behaviours (Bender and Friedman, 2018). Consequently, as predictive models learn from the data distribution they have seen, they tend to favor the demographic group most represented in their training data (Hovy and Spruit, 2016; Shah et al., 2020). This brings serious social consequences as well, since the people who are more likely to be underrep- resented within datasets are those whose representation is often less accounted for within our society. A case in point regards the gender data gap.1 In fact, studies on speech taggers (Hovy and Søgaard, 2015) and speech recognition (Tatman, 2017) showed that the underrepresentation of female speakers in the training data leads to significantly lower accuracy in modeling that demographic group. The problem of gender-related differences has also been inspected within automatic translation, both from text (Vanmassenhove et al., 2018) and from audio (Bentivogli et al., 2020). These studies – focused on the translation of spoken language – revealed a systemic gender bias whenever systems are required to overtly and formally express speaker’s gender in the target languages, while …

Content truncated to 500 words