Score Combination for Improved Parallel Corpus Filtering for Low Resource Conditions

Proceedings of the 5th Conference on Machine Translation (WMT), pages 947–951 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 947 Score Combination for Improved Parallel Corpus Filtering for Low Resource Conditions Muhammad ElNokrashy 1, Amr Hendy1, Mohamed Abdelghaffar1, Mohamed Afify1, Ahmed Tawfik1 and Hany Hassan Awadalla2 1 Microsoft Egypt Development Center, Cairo, Egypt 2 Microsoft Corporation, Redmond, WA, USA {muhammad.elnokrashy,a-amhend,mohamed.abdelghaar,mafify,atawfik,hanyh}@microsoft.com Abstract This paper presents the description of our sub- mission to WMT20 sentence filtering task. We combine scores from custom LASER built for each source language, a classifier built to dis- tinguish positive and negative pairs and the original scores provided with the task. For the mBART setup, provided by the organizers, our method shows 7% and 5% relative improve- ment, over the baseline, in sacreBLEU score on the test set for Pashto and Khmer respec- tively. 1 Introduction Neural machine translation (NMT) brings signifi- cant gains to the field of machine translation. How- ever, it is known to be very sensitive to the qual- ity of parallel data (Khayrallah and Koehn, 2018). This becomes a serious problem when using large but very noisy corpora for training. There is a lot of work on filtering noisy parallel data. For example, the work in (Junczys-Dowmunt, 2018) provides excellent results for large corpora. Low resource languages are even more challenging and the re- sults in (Chaudhary et al., 2019) using multilingual embeddings are very encouraging. This paper describes the system submitted to the WMT20 Shared Task on Parallel Corpus Filtering for Low Resource Conditions. Due to time limita- tion our submission covers only sentence pair fil- tering. However, some of the proposed techniques could be used for sentence alignment and filtering. The task focuses on the Pashto-English and Khmer- English language pairs. It is required that the par- ticipants calculate scores to sort very noisy parallel sentence pairs provided for each language. The top scoring pairs leading to 5M tokens on the English side are then used to train machine translation for each language pair. The organizers also provide LASER-based scores as a baseline according to the method in (Chaudhary et al., 2019), with possi- bly some modifications. We describe our general system architecture followed by development ex- periments to evaluate the merit of different methods and finally report the performance of our best setup per language, using the fairseq recipe provided for the task for both full training (from scratch), and finetuning (mBART) settings. mBART (Liu et al., 2020) is a recently proposed pretraining method. Models initialized using mBART for both language pairs are provided with the task. Our proposed approach combine the scores pro- vided by the organizers with the following two scores: • Margin distance calculated based on custom LASER built for each language using parallel data and a large amount of forward translated mono data provided by the organizers. • Cosine distance between the embeddings gen- erated by a classifier taking the pretrained LASER as input and trained to distinguish parallel and non-parallel sentence pairs using parallel …

Content truncated to 500 words