Volctrans Parallel Corpus Filtering System for WMT 2020

Proceedings of the 5th Conference on Machine Translation (WMT), pages 985–990 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 985 Proceedings of the 5th Conference on Machine Translation (WMT), pages 983–988 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 983 Volctrans Parallel Corpus Filtering System for WMT 2020 Runxin Xu1,3, Zhuo Zhi2,3, Jun Cao3, Mingxuan Wang3, Lei Li3 1 Institute of Computational Linguistic, Peking University 2 Department of Computer Science and Engineering, University of California San Diego 3 ByteDance AI Lab, Shanghai, China runxinxu@gmail.com zzhi@ucsd.edu {caojun.sh, wangmingxuan.89, lileilab}@bytedance.com Abstract In this paper, we describe our submissions to the WMT20 shared task on parallel corpus fil- tering and alignment for low-resource condi- tions. The task requires the participants to align potential parallel sentence pairs out of the given document pairs, and score them so that low-quality pairs can be filtered. Our sys- tem, Volctrans, is made of two modules, i.e., a mining module and a scoring module. Based on the word alignment model, the mining mod- ule adopts an iterative mining strategy to ex- tract latent parallel sentences. In the scoring module, an XLM-based scorer provides scores, followed by reranking mechanisms and ensem- ble. Our submissions outperform the baseline by 3.x/2.x and 2.x/2.x for km-en and ps-en on From Scratch/Fine-Tune conditions. 1 Introduction With the rapid development of machine translation, especially Neural Machine Translation (NMT) (Vaswani et al., 2017; Ott et al., 2018; Zhu et al., 2020), parallel corpus in high quality and large quantity is in urgent demand. These parallel cor- pora can be used to train and build robust machine translation models. However, for some language pairs on low-resource conditions, few parallel re- sources are available. Since it is much easier to obtain quantities of monolingual data, it may help if we can extract parallel sentences from monolin- gual data through alignment and filtering. The WMT19 shared task on parallel corpus fil- tering for low-resource conditions (Koehn et al., 2019) provides noisy parallel corpora in Sinhala- English and Nepali-Englis crawled from the web. Participants are asked to score sentence pairs so that low-quality sentences are filtered. In this year, the WMT20 shared task on parallel Corpus filter- ing and alignment for low-resource conditions is very similar, except that the language pairs become Khmer-English and Pashto-English, and the pro- vided raw data are documents in pair, which require sentence-level alignment. Besides, no data in simi- lar languages are provided for this year. The participants are required to align sentences within documents in different languages and pro- vide a score for each sentence pair. To evaluate the quality of the extracted sentence pairs, they are subsampled to 5 million English words and used to train a neural machine translation model. Finally, the BLEU score of the machine translation system is used to reflect the quality of the sentence pairs. In this paper, we propose the Volctrans filtering system, which consists of a mining module and a scoring module. First, the mining module ex- tracts and aligns potential parallel sentence …

Content truncated to 500 words