The IPN-CIC team system submission for the WMT 2020 similar language task

Proceedings of the 5th Conference on Machine Translation (WMT), pages 409–413 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 409 The IPN-CIC team system submission for the WMT 2020 similar language task Luis A. Men´endez-Salazar1, Grigori Sidorov1, Marta R. Costa-Juss`a2 1Centro de Investigacion en Computacion Instituto Politecnico Nacional, Mexico 2TALP Research Center Universitat Politecnica de Catalunya, Barcelona menendez.sla@gmail.com, sidorov@cic.ipn.mx, marta.ruiz@upc.edu Abstract This paper describes the participation of the NLP research team of the IPN Computer Re- search center in the WMT 2020 Similar Lan- guage Translation Task. We have submitted systems for the Spanish-Portuguese language pair (in both directions). The three submitted systems are based on the Transformer architec- ture and used fine tuning for domain Adapta- tion. 1 Introduction In this paper we describe the Neural Machine Trans- lation(NMT) systems developed by the NLP team of the Computer Research System of the Instituto Polit´ecnico Nacional, M´exico for the similar lan- guages translation shared task of the EMNLP 2020 fifth conference on machine translation (WMT 20). For this task, we submit systems for both di- rections of the Spanish-Portuguese language pair, all the submitted systems were based in a trans- former architecture with a fine tuning for domain adaptation, the difference of the submitted systems was mainly the kind of tokens used (words and sub-word units) and the initialization of the word embeddings in the systems using either a random initialization or pre-trained word embeddings. In the past year task, the submissions of the MLLP-UPV team (Baquero-Arnal et al., 2019) showed that the use of fine tuning was proven to be useful in this specific task, although the test corpus of this year had a higher complexity, the use of fine tuning for context adaptation was also beneficial for translation quality. The paper was organized in the following way. Section 2 describes the architecture used for the systems trained, the section 3 describes the corpora used and the pre-processing of the texts, the section 4 gives a description of the training of the system, section 5 gives a description of the method used for obtaining the translations from the trained system, section 6 shows the results for both the internal evaluation and the evaluation of the task, section 7 is a discussion about the impact of pre-trained word embeddings in the submitted systems and section 8 presents the conclusions. 1.1 Transformer models in NMT Before transformers most state-of-the-art MT sys- tems relied on recurrent neural networks, with at- tention mechanisms, but the RNN based architec- tures although that in theory the information of each token can propagate arbitrary far down in the sequence, due to vanishing gradient in prac- tice when dealing with long sentences or sequences information about the initial tokens can be lost. As a solution of that problem transformers (Vaswani et al., 2017) are an architecture based in an encoder -decoder approach, but rely mainly in self attention mechanisms. 1.1.1 Encoder Each encoder layer consists of two components: a self-attention mechanism and a feed-forward neu- ral …

Content truncated to 500 words