TransQuest at WMT2020: Sentence-Level Direct Assessment

Proceedings of the 5th Conference on Machine Translation (WMT), pages 1049–1055 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 1049 TransQuest at WMT2020: Sentence-Level Direct Assessment Tharindu Ranasinghe♦, Constantin Or˘asan♥ and Ruslan Mitkov♦ ♦Research Group in Computational Linguistics, University of Wolverhampton, UK ♥Centre for Translation Studies, University of Surrey, UK {t.d.ranasinghehettiarachchige, r.mitkov}@wlv.ac.uk c.orasan@surrey.ac.uk Abstract This paper presents the team TransQuest’s participation in Sentence-Level Direct Assessment shared task in WMT 2020. We introduce a simple QE framework based on cross-lingual transformers, and we use it to implement and evaluate two different neural architectures. The proposed methods achieve state-of-the-art results surpassing the results obtained by OpenKiwi, the baseline used in the shared task. We further fine tune the QE framework by performing ensemble and data augmentation. Our approach is the winning solution in all of the language pairs according to the WMT 2020 official results. 1 Introduction The goal of quality estimation (QE) systems is to determine the quality of a translation without having access to a reference translation. This makes it very useful in translation workflows where it can be used to determine whether an automatically translated sentence is good enough to be used for a given purpose, or if it needs to be shown to a human translator for translation from scratch or postediting (Kepler et al., 2019). Quality estimation can be done at different levels: document level, sentence level and word level (Ive et al., 2018). This paper presents TransQuest, a sentence-level quality estimation framework which is the winning solution in all the language pairs in the WMT 2020 Sentence-Level Direct Assessment shared task (Specia et al., 2020). In the past, high preforming quality estimation systems such as QuEst (Specia et al., 2013) and QuEst++ (Specia et al., 2015) were heavily dependent on linguistic processing and feature engineering. These features were fed into traditional machine-learning algorithms like support vector regression and randomised decision trees (Specia et al., 2013), which then determined the quality of a translation. Even though, these approaches provide good results, they are no longer the state of the art, being replaced in recent years by neural-based QE systems which usually rely on little or no linguistic processing. For example the best-performing system at the WMT 2017 shared task on QE was POSTECH, which is purely neural and does not rely on feature engineering at all (Kim et al., 2017). In order to achieve high results, approaches such as POSTECH require extensive pre-training, which means they depend on large parallel data and are computationally intensive (Ive et al., 2018). TransQuest, our QE framework removes this dependency on large parallel data by using crosslingual embeddings (Ranasinghe et al., 2020) that are already fine-tuned to reflect properties between languages (Ruder et al., 2019). Ranasinghe et al. (2020) show that by using them, TransQuest eases the burden of having complex neural network architectures, which in turn entails a reduction of the computational resources. That paper also shows that TransQuest performs well in transfer learning settings where it can be …

Content truncated to 500 words