The ADAPT System Description for the WMT20 News Translation Task

Proceedings of the 5th Conference on Machine Translation (WMT), pages 262–268 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 262 The ADAPT System Description for the WMT20 News Translation Task Venkatesh Balavadhani Parthasarathy‡, Akshai Ramesh‡, Rejwanul Haque and Andy Way The ADAPT Centre, ‡School of Computing Dublin City University, Dublin, Ireland venkatesh.balavadhaniparthasa2,akshai.ramesh2@mail.dcu.ie rejwanul.haque,andy.way@adaptcentre.ie Abstract This paper describes the ADAPT Centre’s submissions to the WMT20 News translation shared task for English-to-Tamil and Tamil-to- English. We present our machine translation (MT) systems that were built using the state-of- the-art neural MT (NMT) model, Transformer. We applied various strategies in order to im- prove our baseline MT systems, e.g. monolin- gual sentence selection for creating synthetic training data, mining monolingual sentences for adapting our MT systems to the task, hy- perparameters search for Transformer in low- resource scenarios. Our experiments show that adding the aforementioned techniques to the baseline yields an excellent performance in the English-to-Tamil and Tamil-to-English transla- tion tasks. 1 Introduction The ADAPT Centre participated in the News Trans- lation Shared Task of the Fifth Conference of Machine Translation (WMT20) in the English-to- Tamil and Tamil-to-English language directions. To build our neural MT systems we used the Transformer model (Vaswani et al., 2017). Our strategies to build the competitive MT systems for the task include applying the state-of-the-art data augmentation approaches (e.g. (Sennrich et al., 2016a; Caswell et al., 2019)), selecting “pseudo” in- domain monolingual sentences for the creation of synthetic bitexts, mining monolingual source and target sentences for the adaptation of neural MT (NMT) systems, finding the optimal set of hyper- parameters for Transformer as far as low-resource translation is concerned. The remainder of the paper is organized as fol- lows. In Section 2, we present our approaches for the MT system building. Section 3 first presents details of the data sets used and then presents the evaluation results with some discussions, while Section 4 concludes our work with avenues for future work. 2 Our Strategies to improve MT Systems 2.1 Data Augmentation The data augmentation methods (Sennrich et al., 2016a; Zhang and Zong, 2016; Burlot and Yvon, 2018; Poncelas et al., 2018; Bogoychev and Sen- nrich, 2019; Caswell et al., 2019; Chen et al., 2019), which usually employ the unlabeled monolingual data in addition to limited bitexts, can positively impact the MT system’s performance and are very popular among the MT developers and researchers (Barrault et al., 2019). In other words, use of augmented bitexts that include synthetic data to improve a NMT system is nowadays a common practice, especially in the under-resource scenar- ios. The synthetic training data whose target-side sentences are original is more effective for domain text translation and generation of fluent transla- tions. In this task, in order to improve our baseline Transformer models, we augmented our training data with the target-original synthetic data. As in Caswell et al. (2019), in order to let the NMT model know that the given source is synthetic, we tag the back-translated source sentences with an extra token. …

Content truncated to 500 words