Data Processing Matters: SRPH-Konvergen AI’s Machine Translation System for WMT’21

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 431–438 November 10–11, 2021. ©2021 Association for Computational Linguistics 431 Data Processing Matters: SRPH-Konvergen AI’s Machine Translation System for WMT’21 Lintang Sutawika∗ Konvergen AI Jakarta, Indonesia lintang@konvergen.ai Jan Christian Blaise Cruz∗ Samsung Research Philippines Manila, Philippines jcb.cruz@samsung.com Abstract In this paper, we describe the submission of the joint Samsung Research Philippines- Konvergen AI team for the WMT’21 Large Scale Multilingual Translation Task - Small Track 2. We submit a standard Seq2Seq Trans- former model to the shared task without any training or architecture tricks, relying mainly on the strength of our data preprocessing tech- niques to boost performance. Our final sub- mission model scored 22.92 average BLEU on the FLORES-101 devtest set, and scored 22.97 average BLEU on the contest’s hidden test set, ranking us sixth overall. Despite us- ing only a standard Transformer, our model ranked first in Indonesian → Javanese, show- ing that data preprocessing matters equally, if not more, than cutting edge model architec- tures and training techniques. 1 Introduction This paper describes the machine translation sys- tem submitted by the joint team of Samsung Research Philippines and Konvergen AI for the WMT’21 Large Scale Multilingual Translation Task. Our team participated in Small Track #2, where the task is to produce a multilingual ma- chine translation system for five Southeast-Asian languages: Javanese, Indonesian, Malay, Tagalog, and Tamil1, plus English, in all 30 directions. We will first describe the filtering heuristics that we used to preprocess the data, and then outline the steps we took to train and evaluate our models. Specific hyperparameters, preprocessing decisions, and other training parameters will be listed in their corresponding sections. Finally, we report our re- sults on the FLORES-101 (Goyal et al., 2021) de- vtest set, as well as on the competition’s hidden test set. ∗Equal contribution. Order determined via coinflip. 1Tamil is considered an official language in Singapore, a Southeast Asian country 2 Parallel Text Preprocessing Heuristics The contest dataset comprises of vari- ous bitext sources, including: bible-uedin (Christodouloupoulos and Steedman, CCAligned (El-Kishky et al., 2020),2015), ELRC 29222, MultiCCAligned (El-Kishky et al., 2020), ParaCrawl 3, TED2020 (Reimers and Gurevych, 2020), WikiMatrix (Schwenk et al., 2019), tico-19, Ubuntu, OpenSubtitles, QED, Tanzil, Tatoeba, GlobalVoices, GNOME, KDE4, and WikiMedia (Tiedemann, 2012). We preprocess the datasets before training in or- der to minimize spurious relations that originate from incorrect text pairs. Our preprocessing re- moves samples based on a few heuristics that we developed based on our observation on the datasets. Each bitext file is applied a different set of pre- processing based on observation. For example we filter by number content for datasets such as CCAAligned while TED2020 is not applied that same filter. In this section, we will cover the decisions made during preprocessing. We observe a score increase of 1.91 BLEU on our submission model when the preprocessing is applied. We report the total num- ber of lines filtered from the bitext for all language pairs on Table 1. 2.1 Filter by …

Content truncated to 500 words