The Fujitsu DMATH Submissions for WMT21 News Translation and Biomedical Translation Tasks

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 162–166 November 10–11, 2021. ©2021 Association for Computational Linguistics 162 The Fujitsu DMATH submissions for WMT21 News Translation and Biomedical Translation Tasks Ander Martínez Fujitsu, Ltd. 4-1-1, Kamikodanaka, Nakahara-ku, Kawasaki 211-8588, Japan ander@fujitsu.com Abstract This paper describes the Fujitsu DMATH sys- tems used for WMT 2021 News Translation and Biomedical Translation tasks. We focused on low-resource pairs, using a simple system. We conducted experiments on English-Hausa, Xhosa-Zulu and English-Basque, and submit- ted the results for Xhosa→Zulu in the News Translation Task, and English→Basque in the Biomedical Translation Task, abstract and ter- minology translation subtasks. Our system combines BPE dropout, sub-subword features and back-translation with a Transformer (base) model, achieving good results on the evaluation sets. 1 Introduction WMT has been exploring the state of the art in MT for many years, and, particularly in recent edi- tions, the participants have shown impressive re- sults. However, often times, these results require very heavy or complex systems, trained on dozens of GPUs. Participants compete for a margin that places them above the rest, combining multiple methods from the latest research. In recent years, different variants of the Trans- former (Vaswani et al., 2017) architecture have been popular for NMT, so can be seen when inspect- ing the submissions to previous editions of WMT. In our systems, we use the Transformer base con- figuration, the smaller one. Our implementation is based on Sockeye 2 (Hieber et al., 2020; Domhan et al., 2020). We combine several techniques or strategies for low-resource pairs. These techniques are described in Section 2. We conducted a few experiments on language pairs Xhosa-Zulu and English-Hausa, from the News Transltion task, and on English-Basque, from the Biomedical Translation task. The results of our experiments are shown in Section 3. 2 Techniques This section describes the strategies used for our NMT models. The first two, bpe dropout and sub- subword features, were used in all the subtasks, while the last one was only used for the biomedical translation subtasks. 2.1 BPE dropout BPE dropout (Provilkov et al., 2020) was intro- duced as an alternative to Kudo (2018). Provilkov et al. found that the main drawback to the subword regularization method is its complexity, since it requires training a unigram language model and uses uses EM and Viterbi algorithms to sample segmentations. BPE dropout works on BPE vocabulary models (Sennrich et al., 2016b), that is, the vocabularies are built in the same way as vanilla BPE. While the unigram language model subword regulariza- tion method uses a statistical model and dynamic programming to be able to sample different seg- mentations from the same sequence, BPE dropout uses random noise to discard certain merges, ran- domly generating a different sequence of subwords each time. This is so because BPE does not store the frequencies of each subword, only the order of the merges. Merges are discarded with a prob- ability p, which is usually 0.1. Provilkov et al. concluded through several experiments that …

Content truncated to 500 words