HW-TSC’s Submission for the WMT22 Efficiency Task

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 677 - 681 December 7-8, 2022 ©2022 Association for Computational Linguistics HW-TSC’s Submissions to the WMT22 Efficiency Task Hengchao Shang1, Ting Hu2, Daimeng Wei1, Zongyao Li1, Xianzhi Yu2, Jianfei Feng2, Jinlong Yang1, Zhiqiang Rao1, Ting Zhu1, Zhengzhe Yu1, Lizhi Lei1, Shimin Tao1, Hao Yang1, Ying Qin1 1Huawei Translation Service Center, Beijing, China 2Huawei Noah’s Ark Lab, Hong Kong, China {shanghengchao,huting35,weidaimeng,lizongyao,yuxianzhi, fengjianfei1,yangjinlong7,raozhiqiang,zhuting20, yuzhengzhe,leilizhi,taoshimin,yanghao30,qinying}@huawei.com Abstract This paper presents the submissions of Huawei Translation Services Center (HW-TSC) to WMT 2022 Efficiency Shared Task. For this year’s task, we still apply sentence-level distil- lation strategy to train small models with differ- ent configurations. Then, we integrate the av- erage attention mechanism into the lightweight RNN model to pursue more efficient decoding. We add a retrain step to our 8-bit and 4-bit mod- els to achieve a balance between model size and translation quality. We still use Huawei Noah’s Bolt1 for INT8 inference and 4-bit stor- age. With Bolt’s support for batch inference and multi-core parallel computing, we finally submit models with different configurations to the CPU latency and throughput tracks to ex- plore the Pareto frontiers. 1 Introduction Transformer and its variants (Vaswani et al., 2017; Shaw et al., 2018; So et al., 2019; Dehghani et al., 2019) have become benchmark models for machine translation. A lot of innovations and engineering optimizations (Tay et al., 2020) in this area are based on Transformer. However, with the increase of bilingual and monolingual data sizes used for training, the size of the model expands and the requirement of computing ability become higher. Taking T5 (Raffel et al., 2020), GPT3 (Brown et al., 2020) and a series of subsequent large models (Fe- dus et al., 2021; Smith et al., 2022) as examples, al- though they have achieved very good performances, it is still difficult for ordinary practitioners to repro- duce or use these models for research and industry application. Especially in scenarios where hard- ware capability is limited, models that balance size, quality and power consumption is urgently needed. The WMT Efficiency task is performed under such constraints. 1https://github.com/huawei-noah/bolt In this year’s task, we still focus on CPU infer- ence optimization and participate in CPU latency and multi-core throughput tracks. We employ knowledge distillation (Hinton et al., 2015) to train small models. The teacher models and distillation data come from official website. We only perform simple data cleaning, and all of our experiments are conducted based on fariseq (Ott et al., 2019). Deep encoder and shallow deocder models can balance quality and inference speed (Wang et al., 2019). We follow this configuration for pursuing extreme efficient decoding. Inspired by SRU++ (Lei, 2021) and AAN (Zhang et al., 2018), we in- tegrate the average attention mechanism with a lightweight RNN for more efficient decoding. We retrain our 8-bit quantisation model (Jacob et al., 2018), then compare its result with that of direct post-quantization (Sung et al., 2015) model. We finally find that in the distillation scenario, the difference …

Content truncated to 500 words