Monotonic Simultaneous Translation with Chunk-wise Reordering and Refinement

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 144–153 November 10–11, 2021. ©2021 Association for Computational Linguistics 144 Illinois Japanese ↔ English News Translation for WMT 2021 Giang Le and Shinka Mori and Lane Schwartz Department of Linguistics, University of Illinois Urbana-Champaign gianghl2@illinois.edu, shinkam2@illinois.edu, lanes@illinois.edu Abstract This system paper describes an end-to- end NMT pipeline for the Japanese ↔ English news translation task as submit- ted to WMT 2021, where we explore the efficacy of techniques such as to- kenizing with language-independent and language-dependent tokenizers, normaliz- ing by orthographic conversion, creating a politeness-and-formality-aware model by implementing a tagger, back-translation, model ensembling, and n-best reranking. We use parallel corpora provided by WMT 2021 organizers for training, and devel- opment and test data from WMT 2020 for evaluation of different experiment mod- els. The preprocessed corpora are trained with a Transformer neural network model. We found that combining various tech- niques described herein, such as language- independent BPE tokenization, incorporat- ing politeness and formality tags, model ensembling, n-best reranking, and back- translation produced the best translation models relative to other experiment sys- tems. 1 Introduction Despite recent advances in machine transla- tion made possible by neural networks with attention mechanism (Bahdanau et al., 2014; Luong et al., 2015), the Japanese-English pair remains a challenging language pair for ma- chine translation systems to handle. Chal- lenges posed by this language pair are multi- faceted, starting from seemingly trivial differ- ences in orthographic representations to deep structural divergence in syntax. This paper describes an end-to-end neural machine trans- lation system and related experiments dedi- cated to the News Translation Shared Task where the target language pair is Japanese ↔ English, as part of a submission to the Sixth Conference on Machine Translation - WMT 2021. In our experiments, we explored the efficacy of techniques such as tokeniz- ing with language-independent and language- dependent tokenizers, normalizing by ortho- graphic conversion, creating a politeness-and- formality-aware model by implementing a tag- ger, back-translation, model ensembling, and n-best reranking. We found that normalizing the text by orthographic conversion did not improve over the baseline but controlling for politeness and formality levels of the text in- creased BLEU by 1.2 points for the En→Ja direction, and other techniques such as back- translation, model ensembling, n-best rerank- ing also produced improvements. The paper gives a detailed review of prior work, with a particular focus on WMT 2020 submissions, and then proceeds to describe our data, model architecture, experiments, results, and discussion of their implications. 2 Prior Work In this section, techniques and development in neural machine translation will be reviewed with a focus on the techniques and implemen- tation most recently used for the Japanese- English language pair. General techniques de- ployed across papers submitted to WMT 2020 are bitext data filtering, back-translation, fine tuning with in-domain data, knowledge dis- tillation, rule-based reranking, transfer learn- ing, co-reference processing, hyperparameter search, segmenting by subword units, BPE dropout, model ensembling, pre-training with monolingual data, experimenting with differ- ent word segmentation …

Content truncated to 500 words