NMT based Similar Language Translation for Hindi - Marathi

Proceedings of the 5th Conference on Machine Translation (WMT), pages 414–417 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 414 NMT based Similar Language Translation for Hindi - Marathi Vandan Mujadia and Dipti Misra Sharma Machine Translation - Natural Language Processing Lab Language Technologies Research Centre Kohli Center on Intelligent Systems International Institute of Information Technology - Hyderabad vandan.mu@research.iiit.ac.in, dipti@iiit.ac.in Abstract This paper describes the participation of team F1toF6 (LTRC, IIIT-Hyderabad) for the WMT 2020 task, similar language translation. We experimented with attention based recurrent neural network architecture (seq2seq) for this task. We explored the use of different lin- guistic features like POS and Morph along with back translation for Hindi-Marathi and Marathi-Hindi machine translation. 1 Introduction Machine Translation (MT) is the field of Natural Language Processing which aims to translate a text from one natural language (i.e Hindi) to another (i.e Marathi). The meaning of the resulting translated text must be fully preserved as the source text in the target language. For the translation task, different types of ma- chine translation systems have been developed and they are mainly Rule based Machine Translation (RBMT)(Forcada et al., 2011), Statistical Machine Translation (SMT) (Koehn, 2009) and Neural Machine Translation (NMT) (Bahdanau et al., 2014). Statistical Machine Translation (SMT) aims to learn a statistical model to determine the correspon- dence between a word from the source language and a word from the target language. Neural Machine Translation is an end to end approach for automatic machine translation without heavily hand crafted feature engineering. Due to recent advances, NMT has been receiving heavy attention and achieved state of the art performance in the task of language translation. With this work, we intend to check how NMT systems could be used for low resource and similar language machine Data Sents Token Type Hindi (Parallel) 38,246 7.6M 39K Marathi (Parallel) 38,246 5.6M 66K Hindi (Mono) 80M - - Marathi (Mono) 3.2M - - Table 1: Hindi-Marathi WMT2020 Training data translation. This paper describes our experiments for the task of similar language translation of WMT-2020. We focused only on Hindi-Marathi language pair for the translation task (both directions). The origin of these two languages are the same as they are Indo-aryan languages(wikipedia, 2020). Hindi is said to have evolved from Sauraseni Prakrit (wikipedia Hindi, 2020) whereas Marathi is said to have evolved from Maharashtri Prakrit (wikipedia Marathi, 2020). They also have evolved as two major languages in different regions of India. In this work, we focused only on recurrent neural network with attention based sequence to sequence architecture throughout all experiments. Along with it, we also explored the morph(Virpioja et al., 2013) induced sub-word segmentation with byte pair encoding (BPE)(Sennrich et al., 2016b) to en- able open vocabulary translation. We used POS tags as linguistic feature and back translation to leverage synthetic data for machine translation task in both directions. In the similar language transla- tion task of WMT-2020, we participated as team named “f1plusf6”. 2 Data We utilised parallel and monolingual corpora pro- vided …

Content truncated to 500 words