Edinburgh’s Submission to the WMT 2022 Efficiency Task

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 661 - 667 December 7-8, 2022 ©2022 Association for Computational Linguistics Edinburgh’s submission to the WMT 2022 Efficiency task Nikolay Bogoychev† Biao Zhang†, Maximiliana Behnke† Graeme Nail† Jelmer van der Linde† Sidharth Kashyap‡ Kenneth Heafield† †University of Edinburgh ‡Intel Corporation {n.bogoych, biao.zhang, maximiliana.behnke, graeme.nail, jelmer.vanderlinde kenneth.heafield}@ed.ac.uk, sidharth.n.kashyap@intel.com Abstract We participated in all tracks of the WMT 2022 efficient machine translation task: single-core CPU, multi-core CPU, and GPU hardware with throughput and latency conditions. Our sub- missions explore a number of efficiency strate- gies: knowledge distillation, a simpler simple recurrent unit (SSRU) decoder with one or two layers, shortlisting, deep encoder, shallow de- coder, pruning and bidirectional decoder. For the CPU track, we used quantized 8-bit models. For the GPU track, we used FP16 quantisa- tion. We explored various pruning strategies and combination of one or more of the above methods. 1 Introduction This paper describes the University of Edinburgh’s submission to Seventh Conference on Machine Translation (WMT2022) Efficiency Task1, which measures performance on latency and throughput on both CPU and GPU, in addition to translation quality. Our submission focused on the trade-off between these metrics and quality. Our submission builds upon the work of last year’s submission (Behnke et al., 2021). We trained our models in a teacher-student setting (Kim and Rush, 2016), using the data provided by the organ- isers. For the students, we used a Simpler Sim- ple Recurrent Unit (SSRU) (Kim et al., 2019) de- coder, used a target vocabulary shortlist, and ex- perimented with pruning the student models by removing component and block-level parameters to improve speed. We used 8-bit quantisation for the CPU submission and FP16 quantisation for the GPU submission. We further experimented with IBDecoder (Zhang et al., 2020). For running our experiments, we improved upon the Marian (Junczys-Dowmunt et al., 2018) ma- chine translation framework by incorporating speed 1http://statmt.org/wmt22/ efficiency-task.html ups for 8-bit matrix multiplication operations, opti- mizations for pruning neural network parameters on Intel CPUs, and profiler aided optimisation of various components. 1.1 Efficiency Shared Task The WMT22 efficiency shared task consists of two sub-tasks: throughput and latency. Systems should translate English to German under the constrained conditions, where the teacher model and the dis- tilled data are provided. For each task, systems are provided 1 million lines of raw English input with at most 150 space-separated words. The through- put task receives this input directly. The latency task, introduced in WMT21, is fed input one sen- tence at a time, waiting for the translation output before providing the next sentence. Throughput is measured on multi-core CPU or GPU system, and latency is measured on single- core CPU or GPU systems. The CPU-based eval- uations use an Intel Ice Lake system via Oracle Cloud BM.Optimized3.36, while the GPU-based use a single A100 via Oracle Cloud BM.GPU4.8. Entries to both tasks are measured on quality, approximated via COMET score (Rei et al., 2020), speed, model size, Docker image size, and memory consumption. We …

Content truncated to 500 words