Pivot Based Transfer Learning for Neural Machine Translation: CFILT IITB @ WMT 2021 Triangular MT

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 336–340 November 10–11, 2021. ©2021 Association for Computational Linguistics 336 Pivot based Transfer Learning for Neural Machine Translation: CFILT IITB @ WMT 2021 Triangular MT Shivam Mhaskar, Pushpak Bhattacharyya Department of Computer Science and Engineering Indian Institute of Technology Bombay Mumbai, India {shivammhaskar, pb}@cse.iitb.ac.in Abstract In this paper, we discuss the various tech- niques that we used to implement the Russian- Chinese machine translation system for the Triangular MT task at WMT 2021. Neural Machine translation systems based on trans- former architecture have an encoder-decoder architecture, which are trained end-to-end and require a large amount of parallel corpus to produce good quality translations. This is the reason why neural machine translation sys- tems are referred to as data hungry. Such a large amount of parallel corpus is majorly available for language pairs which include En- glish and not for non-English language pairs. This is a major problem in building neural ma- chine translation systems for non-English lan- guage pairs. We try to utilize the resources of the English language to improve the trans- lation of non-English language pairs. We use the pivot language, that is English, to lever- age transfer learning to improve the quality of Russian-Chinese translation. Compared to the baseline transformer-based neural ma- chine translation system, we observe that the pivot language-based transfer learning tech- nique gives a higher BLEU score. 1 Introduction The aim of this work is to improve the quality of Machine Translation (MT) for low-resource, dis- tant and non-English language pairs. One of the major requirements for the good performance of the Neural Machine Translation (NMT) systems is the availability of a large parallel corpus. Such large parallel corpus of good quality is not available for low-resource, distant and non-English language pairs but mostly available for language pairs con- taining English. This poses a major challenge in developing good quality Machine Translation sys- tems for non-English and distant language pairs. As a result there is a need to come up with addi- tional resources by augmenting parallel corpora or by using knowledge from other tasks using transfer learning for translation of non-English language pairs. In this paper, we focus on leveraging the knowledge from other tasks using transfer learning to improve the performance of NMT systems for low resource language pairs. In our pivot based transfer learning experiments we try to utilize the resources of English language, that is English-Chinese and English-Russian par- allel corpora to improve the quality of Russian- Chinese translation. We implement techniques which efficiently use the resources of the English language for the task of Russian-Chinese transla- tion. 2 Related Work Recurrent Neural Network (RNN) based encoder decoder architectures (Bahdanau et al., 2014; Cho et al., 2014; Sutskever et al., 2014) were initially used in NMT systems. Transformer (Vaswani et al., 2017) architecture improved the performance of NMT systems. In order to enable translation be- tween distant and non-English language pairs for which a large amount of parallel corpus is …

Content truncated to 500 words