Infosys Machine Translation System for WMT20 Similar Language Translation Task

Proceedings of the 5th Conference on Machine Translation (WMT), pages 437–441 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 437 Infosys Machine Translation System for WMT20 Similar Language Translation Task Kamalkumar Rathinasamy, Amanpreet Singh, Balaguru Sivasambagupta, Prajna Prasad Neerchal, Vani Sivasankaran Infosys Limited {kamalkumar r,amanpreet.singh04,balaguru.s,prajna.neerchal,vani.s}@infosys.com Abstract This paper describes Infosys’ submission to the WMT20 Similar Language Translation shared task. We participated in Indo-Aryan language pair in the language direction Hindi to Marathi. Our baseline system is byte- pair encoding based transformer model trained with the fairseq sequence modeling toolkit. Our final system is an ensemble of two trans- former models, which ranked first in the WMT20 evaluation. One model is designed to learn the nuances of translation of this low resource language pair by taking advantage of the fact that the source and target languages are the same alphabet languages. The other model is the result of experimentation with the pro- portion of back-translated data to the parallel data to improve translation fluency. 1 Introduction Neural Machine Translation (Bahdanau et al., 2015; Vaswani et al., 2017) is the most popular approach for machine translation. Transformer- based NMT has outperformed many recurrent neu- ral network based models. There is scope for im- provement in NMT, particularly for low-resource language pairs. Our techniques are experimented on the fairseq sequence modeling toolkit (Ott et al., 2019) for NMT. Our system is an ensemble of two transformer-based models. One designed for low- resource language pairs by taking advantage that both are same alphabet languages. The other model is built after experimenting on renowned back- translation technique (Sennrich et al., 2016a) by exploiting target monolingual data. 2 Data Hindi-Marathi bitext data contains ∼49K sentence pairs. Target monolingual data comprises of 326K Newscrawl sentences and 10,839K raw sentences. 2.1 Data Preprocessing Typical training sentence pairs comprises of a source and a target sentence. There are ∼1K train- ing sentence pairs where source or target contains multiple sentences delimited by ‘/’. Matching pair for these sentences is derived based on the prox- imity of token lengths between source and target sentence. Non-printable characters are removed, punctua- tions are normalized, and the data is tokenized, with the Moses tokenizer. Byte-pair encoding (BPE) has been adopted (Sennrich et al., 2016a) to build source and target sub-word vocabularies of size 22.5K and 32.8K respectively, when configured to construct with 60K symbols. 2.2 Data filtering 2.2.1 Bitext data Sentences with more than 175 words, sentences with no words, and sentence pairs exceeding length ratio of 1.5 are removed from training data. This eliminated around 18% of the overall real bitext data. 2.2.2 Synthetic data CommonCrawl n-grams raw monolingual files are processed1 to remove sentences with invalid char- acters, strip leading and trailing whitespaces, and remove duplicate sentences. 3 System Overview Our Hindi-Marathi primary system is an ensem- ble of two transformer models. One is back- translated model and the other model is trained on anonymized data. 1https://github.com/kpu/preprocess 438 3.1 Base Model Architecture and Hyperparameters Our model is built using fairseq2 …

Content truncated to 500 words