Towards Debiasing Translation Artifacts

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3983 - 3991 July 10-15, 2022 ©2022 Association for Computational Linguistics Towards Debiasing Translation Artifacts Koel Dutta Chowdhury1, Rricha Jalota1, Cristina España-Bonet2, and Josef van Genabith1,2 1Saarland University, Saarland Informatics Campus, Germany 2German Research Center for Artificial Intelligence (DFKI) koel.duttachowdhury@uni-saarland.de rrja00001@stud.uni-saarland.de {cristinae, Josef.Van_Genabith}@dfki.de Abstract Cross-lingual natural language processing re- lies on translation, either by humans or ma- chines, at different levels, from translating training data to translating test sets. However, compared to original texts in the same lan- guage, translations possess distinct qualities re- ferred to as translationese. Previous research has shown that these translation artifacts in- fluence the performance of a variety of cross- lingual tasks. In this work, we propose a novel approach to reducing translationese by extend- ing an established bias-removal technique. We use the Iterative Null-space Projection (INLP) algorithm, and show by measuring classifica- tion accuracy before and after debiasing, that translationese is reduced at both sentence and word level. We evaluate the utility of debias- ing translationese on a natural language infer- ence (NLI) task, and show that by reducing this bias, NLI accuracy improves. To the best of our knowledge, this is the first study to de- bias translationese as represented in latent em- bedding space. 1 Introduction “Translationese” refers to features in (professional human or machine) translated text that distin- guishes it from non-translated, original text in the same language. Carriers of translationese include lexical and word order choices that are influenced by the source language (Gellerstam, 1986), as well the use of more explicit and standardised construc- tions (Baker et al., 1993) compared to original text. Translationese has a significant impact in machine translation evaluation. Toral et al. (2018) found that translating source sentences that are already the result of translation are easier to translate than original sentences. Similarly, Edunov et al. (2020) show that back-translation results in large BLEU scores when translationese is on the source side and original text is used as reference. To avoid such artifacts, it is advised to use original source sen- tences for machine translation evaluation (Zhang and Toral, 2019; Graham et al., 2020). Riley et al. (2020) train sentence-level classifiers to differen- tiate translationese from original target text, and then use this classifier to tag the training data for an NMT model to produce output that shows fewer translationese effects. Translation-inherited arti- facts have been shown to have significant impact on other tasks as well. For example, Singh et al. (2019) show that substituting segments of origi- nal training samples by their translations from an- other language improves performance on natural language inference (NLI) tasks. Clark et al. (2020) introduce a translation-free Question Answering dataset to avoid having inflated gains from trans- lation artifacts in transfer-learning tasks. Artetxe et al. (2020) show that cross-lingual models suffer from induced translation artifacts when evaluated on translated test sets. These examples motivate the need for reducing translation artifacts. …

Content truncated to 500 words