Comparing Feature-Engineering and Feature-Learning Approaches for Multilingual Translationese Classification

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8596–8611 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8596 Comparing Feature-Engineering and Feature-Learning Approaches for Multilingual Translationese Classification Daria Pylypenko*1, Kwabena Amponsah-Kaakyire*1,2, Koel Dutta Chowdhury*1, Josef van Genabith1,2, and Cristina España-Bonet2 1Saarland University, 2German Research Center for Artificial Intelligence (DFKI) Saarland Informatics Campus, Saarbrücken, Germany {daria.pylypenko,koel.duttachowdhury}@uni-saarland.de kwabena.amponsah-kaakyire@dfki.de {cristinae, Josef.Van_Genabith}@dfki.de Abstract Traditional hand-crafted linguistically- informed features have often been used for distinguishing between translated and original non-translated texts. By contrast, to date, neural architectures without manual feature engineering have been less explored for this task. In this work, we (i) compare the traditional feature-engineering-based ap- proach to the feature-learning-based one and (ii) analyse the neural architectures in order to investigate how well the hand-crafted features explain the variance in the neural models’ predictions. We use pre-trained neural word embeddings, as well as several end-to-end neural architectures in both monolingual and multilingual settings and compare them to feature-engineering-based SVM classifiers. We show that (i) neural architectures out- perform other approaches by more than 20 accuracy points, with the BERT-based model performing the best in both the monolingual and multilingual settings; (ii) while many individual hand-crafted translationese features correlate with neural model predictions, fea- ture importance analysis shows that the most important features for neural and classical architectures differ; and (iii) our multilingual experiments provide empirical evidence for translationese universals across languages. 1 Introduction Texts originally written in a language exhibit prop- erties that distinguish them from texts that are the result of a translation into the same language. These properties are referred to as translationese (Gellerstam, 1986). Earlier studies have shown that using various hand-crafted features for super- vised learning can be effective for translationese classification (Baroni and Bernardini, 2005; Volan- sky et al., 2015; Rubino et al., 2016). However, this approach has a number of limitations. Firstly, *Equal contribution. manually designed features may be partial and non- exhaustive in a sense that they are based on our linguistic intuitions, and thus may not be guaran- teed to capture all discriminative characteristics of the input data seen during training. Other limita- tions are related to the difficulties in obtaining lin- guistic annotation tools (e.g., parsers, taggers, etc.) for some languages, reliance on n-gram counts, limited contexts, corpus specific characteristics, among others. In this work, we compare a stan- dard approach based on hand-crafted features with automatic feature learning based on data, task and learner without prior linguistic assumptions. Moreover, most previous approaches have fo- cused on classifying translationese in the monolin- gual setting, i.e. translations come from one or mul- tiple source languages, but the language on which to perform the classification is always the same. To the best of our knowledge, the multilingual set- ting with multiple source and target languages has not been explored yet. If translationese features are language-independent or shared among lan- guages, multilingual translationese classification experiments would show the effect. We perform bi- nary translationese classification not only in mono- , but also …

Content truncated to 500 words