NAIST-NICT-TIT WMT22 General MT Task Submission

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 244 - 250 December 7-8, 2022 ©2022 Association for Computational Linguistics NAIST-NICT-TIT WMT22 General MT Task Submission Hiroyuki Deguchi†,†† Kenji Imamura†† Masahiro Kaneko††† Yuto Nishida† Yusuke Sakai† Justin Vasselli† Huy Hien Vu† Taro Watanabe† † Nara Institute of Science and Technology ††† Tokyo Institute of Technology †† National Institute of Information and Communications Technology †{deguchi.hiroyuki.db0, nishida.yuto.nu8, sakai.yusuke.sr9, vasselli.justin_ray.vk4, vu.huy_hien.va9, taro}@is.naist.jp ††kenji.imamura@nict.go.jp, †††masahiro.kaneko@nlp.c.titech.ac.jp Abstract In this paper, we describe our NAIST-NICT- TIT submission to the WMT22 general ma- chine translation task. We participated in this task for the English ↔ Japanese language pair. Our system is characterized as an ensemble of Transformer big models, k-nearest-neighbor machine translation (kNN-MT) (Khandelwal et al., 2021), and reranking. In our translation system, we construct the datastore for kNN-MT from back-translated monolingual data and integrate kNN-MT into the ensemble model. We designed a reranking system to select a translation from the n-best translation candidates generated by the trans- lation system. We also use a context-aware model to improve the document-level consis- tency of the translation. 1 Introduction We participated in the WMT22 general ma- chine translation task in two language direc- tions, English-to-Japanese (En-Ja) and Japanese- to-English (Ja-En). We built our system on an ensemble of Transformer big models, k-nearest- neighbor machine translation (kNN-MT) (Khan- delwal et al., 2021), and reranking. Figure 1 shows an overview of our system. Our translation system is a combination of kNN-MT and an ensemble of four Transformer big models. We train each of the Transformer NMT models using a different random seed, and pick one model as kNN-MT. A notable point about our system is that we construct the datastore for kNN-MT from back-translated monolingual data rather than reusing training data. We found that us- ing back-translated data improves translation per- formance compared with using a parallel training corpus for the datastore. Our reranker is designed to select the transla- tion from the n-best translation candidates gener- ated by the translation system. The reranker com- Decoding Training / Offline Process Ensemble kNN-MT Reranking Target-side Monolingual Data Test Data System Output Training Parallel Data Training Parallel Data Synthetic Parallel Data kNN-MT Datastore Source→Target Target→Source Source→Target (L2R) Source→Target (R2L) Base MT Score Target→Source (L2R) Target→Source (R2L) N-best Candidates … Figure 1: System overview. putes the weighted sum of each translation candi- date across multiple models and selects the trans- lation candidate with the highest score. We used k-best batch MIRA (Cherry and Foster, 2012) to select the weights for the model scores that maxi- mize the BLEU score of the development set. 2 Corpora and Preprocessing For the training data, we used all the provided bilingual parallel data: JParaCrawl v3 (Morishita et al., 2020), News Commentary v16 (Tiedemann, 2012), Wiki Titles v3, WikiMatrix, Japanese- English Subtitle Corpus (Pryzant et al., 2018), The Kyoto Free Translation Task Corpus, and TED Talks. As the English translation of the Japanese- English Subtitle Corpus is only available in lower- case, we trained a Moses truecaser (Koehn …

Content truncated to 500 words