TenTrans Multilingual Low-Resource Translation System for WMT21 Indo-European Languages Task

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 376–382 November 10–11, 2021. ©2021 Association for Computational Linguistics 376 TenTrans Multilingual Low-Resource Translation System for WMT21 Indo-European Languages Task Han Yang1,2†, Bojie Hu2, Wanying Xie2, Ambyer Han2, Pan Liu2, Jinan Xu1∗, Qi Ju2∗ 1Beijing Jiaotong University 2TencentMT Oteam {19120421, jaxu}@bjtu.edu.cn {bojiehu, wanyingxie, ambyera, galeliu, damonju}@tencent.com Abstract This paper describes TenTrans’ submission to WMT21 Multilingual Low-Resource Transla- tion shared task for the Romance language pairs. This task focuses on improving trans- lation quality from Catalan to Occitan, Roma- nian and Italian, with the assistance of related high-resource languages. We mainly utilize back-translation, pivot-based methods, multi- lingual models, pre-trained model fine-tuning, and in-domain knowledge transfer to improve the translation quality. On the test set, our best-submitted system achieves an average of 43.45 case-sensitive BLEU scores across all low-resource pairs. Our data, code, and pre- trained models used in this work are available in TenTrans evaluation examples1. 1 Introduction We participate in the WMT21 Multilingual Low- Resource Translation shared task. This task fo- cuses on the multilinguality in the cultural heritage domain for two Indo-European language families: North-Germanic and Romance. We devote the re- search into translations among Romance languages, including Catalan→Occitan, Catalan→Romanian, Catalan→Italian. Additionally, this task explicitly encourages the use of data of four related high- resource languages (Spanish, French, Portuguese and English) in the same linguistic family. For the model architecture, we adopt a universal encoder-decoder architecture that shares parame- ters across all languages (Johnson et al., 2017). And almost all of the subsequent experiments are based on Transformer base model (Vaswani et al., 2017). †This work is done by the author as an intern at TencentMT Oteam. *Corresponding author. 1https://github.com/TenTrans/ TenTrans/blob/master/example/WMT21/ WMT21-low-resource-MNMT.md To effectively exploit low and high resource data in the multilingual low-resource scenario, we ex- plore several approaches, and each approach shows effectiveness. We employ back-translation (Sen- nrich et al., 2016a) and pivot-based methods to augment the training corpus. In terms of knowl- edge transfer, we explore the pre-trained model and the multilingual model that trained with both low and high resource language pairs. Moreover, we extract in-domain corpus by a domain classifier and adapt the model to the target domain by in-domain fine-tuning. This paper is structured as follows: Section 2 introduces used datasets, data statistic and pre- processing pipeline. Section 3 describes the details of different approaches. In Section 4 we present experimental settings and results. Section 5 draws a brief conclusion of our work in the WMT21. 2 Data 2.1 Datasets The training datasets are majorly provided by the publicly available OPUS (Tiedemann, 2012) repos- itory. We use almost all available datasets provided in the task, including Europarl, JW300, WikiMa- trix, MultiCCAligned, OPUS-100, Bible, ELRC, and 167.2K It-Ro pairs in TED talks as well as 15M/360K sentence pairs of En-It/En-Ro extracted from Wikipedia dumps. For datasets that can be found through the resources search form on the top-level website of OPUS, we use opus-tools2 to extract low-resource language pairs. As for rest of the …

Content truncated to 500 words