Two-Phase Cross-Lingual Language Model Fine-Tuning for Machine Translation Quality Estimation

Proceedings of the 5th Conference on Machine Translation (WMT), pages 1024–1028 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 1024 Two-Phase Cross-Lingual Language Model Fine-Tuning for Machine Translation Quality Estimation Dongjun Lee Bering Lab, Republic of Korea djlee@beringlab.com Abstract In this paper, we describe the Bering Lab’s submission to the WMT 2020 Shared Task on Quality Estimation (QE). For word-level and sentence-level translation quality estimation, we fine-tune XLM-RoBERTa, the state-of-the- art cross-lingual language model, with a few additional parameters. Model training consists of two phases. We first pre-train our model on a huge artificially generated QE dataset, and then we fine-tune the model with a human- labeled dataset. When evaluated on the WMT 2020 English-German QE test set, our systems achieve the best result on the target-side of word-level QE and the second best results on the source-side of word-level QE and sentence- level QE among all submissions. 1 Introduction Machine translation quality estimation (QE) is the task of estimating the quality of machine-translated (MT) output given just the source text at various granularity levels (word, sentence, and document) (Fonseca et al., 2019). Word-level QE can be di- vided into target-side and source-side tasks. On the target-side, the goal is to predict whether each word in the MT sentence is OK or BAD and whether there are missing words between each word. The goal on the source-side is to predict whether each word in the source sentence is correctly translated or not. On the other hand, sentence-level QE aims to predict the Human Translation Error Rate (HTER) (Snover et al., 2006) of the MT sentence, which measures the required amount of human editing to fix the MT sentence. In this paper, we propose a cross-lingual lan- guage model fine-tuning approach with a few addi- tional parameters for word-level and sentence-level QE. As a pre-trained cross-lingual language model, we use XLM-RoBERTa (XLM-R) (Conneau et al., 2019), which shows state-of-the-art performance for a wide range of cross-lingual transfer tasks. In addition, since labeling the QE dataset requires a large amount of human labor, we generate and utilize a huge artificial QE dataset to improve the performance of our model. Our contributions are summarized as follows. • We propose an XLM-R-based neural net- work architecture for the QE. Our model can be jointly trained for both word-level and sentence-level QE. • We generate a huge artificial QE dataset based on a parallel corpus with OpenNMT-py (Klein et al., 2017) and the TER tool (Snover et al., 2006). • We train our model in two phases. First, we train our model with a huge artificially gen- erated dataset. Then, we fine-tune the model with a human-labeled dataset. In the experiment using the WMT 2020 English- German word-level QE test set, we achieve an MCC of 0.597 and 0.454 for the target-side and source-side, respectively, showing the best and sec- ond best performance among all submitted systems, respectively. For the sentence-level QE test set, we achieve a Pearson correlation of 0.723, which ranks second …

Content truncated to 500 words