Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task

Proceedings of the 5th Conference on Machine Translation (WMT), pages 191–196 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 191 Speed-optimized, Compact Student Models that Distill Knowledge from a Larger Teacher Model: the UEDIN-CUNI Submission to the WMT 2020 News Translation Task Ulrich Germann,⋆ Roman Grundkiewicz,⋆ Martin Popel,‡ Radina Dobreva,⋆ Nikolay Bogoychev,⋆ Kenneth Heafield⋆ ⋆ University of Edinburgh, UK ‡ Charles University, Prague, Czech Republic {ugermann|rgrundki|rdobreva|nbogoych|kheafiel}@inf.ed.ac.uk popel@ufal.mff.cuni.cz Abstract We describe the joint submission of the University of Edinburgh and Charles Uni- versity, Prague, to the Czech/English track in the WMT 2020 Shared Task on News Translation. Our fast and compact stu- dent models distill knowledge from a larger, slower teacher. They are designed to offer a good trade-off between translation quality and inference efficiency. On the WMT 2020 Czech ↔ English test sets, they achieve translation speeds of over 700 whitespace- delimited source words per second1 on a sin- gle CPU thread, thus making neural transla- tion feasible on consumer hardware without a GPU. 1 Introduction The conventional set-up of the WMT Shared Tasks on News Translation emphasizes translation qual- ity (however measured) above all else. Constraints on the data that may be used for training in the ‘constrained’ track establish a level playing field in terms of the information available to the trans- lation model and its training process, but there are no constraints on the computational power and effort spent to achieve the results. In contrast, the WNGT Shared Task on Efficient Translation (Heafield et al., 2020) encourages participants to submit systems that are both accurate and efficient during inference (i.e., translation). So far, there has been little interaction between the two tasks. With our joint submission between the University of Edinburgh (UEDIN) and Charles University, Prague (CUNI), we strive to bridge this gap. We submitted small, efficient systems that distilled knowledge from a more powerful teacher model via sequence-level knowledge distillation (Kim and 1 Bogoychev et al. (2020) report translation speeds of up to 3135 source words per second on a single CPU thread; the actual throughput depends not only on the computer hardware used for translation but also on the distribution of translation segment lengths in the test set. Rush, 2016). In a nutshell, the procedure can be described as follows: 1. Train a powerful teacher model on the available training data set D. 2. Translate the source side of D plus avail- able monolingual data in the source language and appropriate text domain with the teacher model to generate the training set D′. 3. Train a small student model on D′. While the computational effort to first train a teacher model and then distill its knowledge into a student model is considerably greater than just training the teacher — in addition to training the teacher, we have to translate the training data and then train a student on that data —, the advan- tage is at inference time. Even the larger of the two student models we present in this paper can translate on …

Content truncated to 500 words