The RoyalFlush System for the WMT 2022 Efficiency Task

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 671 - 676 December 7-8, 2022 ©2022 Association for Computational Linguistics The RoyalFlush System for the WMT 2022 Efficiency Task Bo Qin1, Aixin Jia1, Qiang Wang2,1, Jianning Lu1, Shuqin Pan1, Haibo Wang1, Ming Chen1∗ 1RoyalFlush AI Research Institute, Hangzhou, China 2Zhejiang University, Hangzhou, China {qinbo, jiaaixin, wangqiang3}@myhexin.com {lujianning, panshuqin, wanghaibo3, chenming}@myhexin.com Abstract This paper describes the submission of the ROY- ALFLUSH neural machine translation system for the WMT 2022 translation efficiency task. Unlike the commonly used autoregressive trans- lation system, we adopted a two-stage transla- tion paradigm called Hybrid Regression Trans- lation (HRT) to combine the advantages of au- toregressive and non-autoregressive translation. Specifically, HRT first autoregressively gener- ates a discontinuous sequence (e.g., make a prediction every k tokens, k > 1) and then fills in all previously skipped tokens at once in a non-autoregressive manner. Thus, we can easily trade off the translation quality and speed by adjusting k. In addition, by integrat- ing other modeling techniques (e.g., sequence- level knowledge distillation and deep-encoder- shallow-decoder layer allocation strategy) and a mass of engineering efforts, HRT improves 80% inference speed and achieves equivalent translation performance with the same-capacity AT counterpart. Our fastest system reaches 6k+ words/second on the GPU latency setting, es- timated to be about 3.1x faster than the last year’s winner. 1 Introduction Large-scale transformer models have made im- pressive progress in past WMT translation tasks, but it is still challenging for practical model de- ployment due to time-consuming inference speed (Wang et al., 2018b; Li et al., 2019). To build a fast and accurate machine translation system, par- ticipants in past WMT efficiency tasks developed and validated many efficient techniques, such as knowledge distillation (Hinton et al., 2015; Kim and Rush, 2016), light network architecture (Ka- sai et al., 2020), quantization (Lin et al., 2020) etc. We noticed that all the above efforts are aimed at autoregressive translation (AT) models. ∗Corresponding author. In contrast, other translation paradigms, like non- autoregressive translation (NAT) (Gu et al., 2017) or semi-autoregressive translation (SAT) (Wang et al., 2018a) etc., have not been well studied. In this participation, we restrict ourselves to the GPU latency track and attempt to investigate the potential of non-standard translation paradigms. However, replicating the vanilla non-autoregressive or semi-autoregressive models degrades the trans- lation quality severely in our preliminary experi- ments. To this end, we explore hybrid-regressive translation (HRT), the two-stage translation proto- type, to better combine the advantages of autore- gressive and non-autoregressive translation (Wang et al., 2021b). Specifically, HRT first uses an au- toregressive decoder to generate a discontinuous target sequence with the interval k (k > 1). Then, HRT fills the remaining slots at once with a non- autoregressive decoder. The two decoders share the same parameters without adding additional ones. Thus, HRT can easily trade-off between transla- tion quality and speed by adjusting k 1. Please see Table 1 for the comparison between different translation paradigms. In addition to the change of translation paradigm, …

Content truncated to 500 words