Do Humans Translate like Machines? Students’ Conceptualisations of Human and Machine Translation

Tailoring Domain Adaptation for Machine Translation Quality Estimation Javad Pourmostafa Roshan Sharami, Dimitar Shterionov, Fr´ed´eric Blain, Eva Vanmassenhove, Mirella De Sisto, Chris Emmery, Pieter Spronck Department of Cognitive Science and Artificial Intelligence, Tilburg University {j.pourmostafa,d.shterionov,F.L.G.Blain,e.o.j.vanmassenhove, M.DeSisto,C.D.Emmery,p.spronck}@tilburguniversity.edu Abstract While quality estimation (QE) can play an important role in the translation process, its effectiveness relies on the availability and quality of training data. For QE in particular, high-quality labeled data is of- ten lacking due to the high cost and effort associated with labeling such data. Aside from the data scarcity challenge, QE mod- els should also be generalizable; i.e., they should be able to handle data from dif- ferent domains, both generic and specific. To alleviate these two main issues — data scarcity and domain mismatch — this pa- per combines domain adaptation and data augmentation in a robust QE system. Our method first trains a generic QE model and then fine-tunes it on a specific domain while retaining generic knowledge. Our results show a significant improvement for all the language pairs investigated, better cross-lingual inference, and a superior per- formance in zero-shot learning scenarios as compared to state-of-the-art baselines. 1 Introduction Predicting the quality of machine translation (MT) output is crucial in translation workflows. Inform- ing translation professionals about the quality of an MT system allows them to quickly assess the overall usefulness of the generated translations and gauge the amount of post-editing that will be required (Tamchyna, 2021; Murgolo et al., 2022). Quality estimation (QE) is an approach that aims © 2023 The authors. This article is licensed under a Creative Commons 4.0 licence, no derivative works, attribution, CC- BY-ND. to reduce the human effort required to analyze the quality of an MT system by assessing the quality of its output without the need for reference translations. QE can be applied on word-, sentence- or document-levels. The goal of sentence-level QE, which is the focus of our work, is to predict a quality label based on a source sentences and its MT equivalents. This label, (i.e., the quality estimate), can be expressed in various ways such as TER/HTER (Snover et al., 2006), BLEU (Pa- pineni et al., 2002) or any metric of interest to the user. Training a sentence-level QE system typically requires aligned data of the form: source sentence (SRC), target sentence (TRG), and quality gold label (LBL). However, most quality labels are by-products of MT and post-editing — a rather difficult and expensive process — limiting the size of the available QE data (Rei et al., 2020; Zouhar et al., 2023). The WMT QE shared task (Specia et al., 2021; Zerva et al., 2022) has been offered a platform to compare different QE systems and to share QE data. Despite efforts from initiatives like the QE shared task to publicly release QE datasets, such resources remain scarce across language pairs and, by extension, also have a limited coverage across domains (Fomicheva et al., 2020a; Fomicheva et al., 2022). This can pose a challenge for all QE models, …

Content truncated to 500 words