A Hierarchical Network for Abstractive Meeting Summarization with Cross-Domain Pretraining

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1–14 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 1 Fully Quantized Transformer for Machine Translation Gabriele Prato Mila, Universit´e de Montr´eal pratogab@mila.quebec Ella Charlaix Huawei Noah’s Ark Lab ella.charlaix@huawei.com Mehdi Rezagholizadeh Huawei Noah’s Ark Lab mehdi.rezagholizadeh@huawei.com Abstract State-of-the-art neural machine translation methods employ massive amounts of param- eters. Drastically reducing computational costs of such methods without affecting per- formance has been up to this point unsuccess- ful. To this end, we propose FullyQT: an all- inclusive quantization strategy for the Trans- former. To the best of our knowledge, we are the first to show that it is possible to avoid any loss in translation quality with a fully quan- tized Transformer. Indeed, compared to full- precision, our 8-bit models score greater or equal BLEU on most tasks. Comparing our- selves to all previously proposed methods, we achieve state-of-the-art quantization results. 1 Introduction The idea of using neural networks for machine translation was only recently proposed (Kalchbren- ner and Blunsom, 2013; Sutskever et al., 2014; Cho et al., 2014). Nonetheless, the approach became the state-of-the-art in the field (Ahmed et al., 2017; Ott et al., 2018; Edunov et al., 2018). A key element of this success was to allow the decoder to attend to all hidden states of the encoder (Bahdanau et al., 2014). A few variations to this additive attention mecha- nism have been proposed, such as multiplicative and self-attention (Luong et al., 2015; Cheng et al., 2016; Lin et al., 2017). The latter formed the basis of the Transformer network (Vaswani et al., 2017), which achieved state-of-the-art results in machine translation. Inspiring a new wave of work, numer- ous natural language processing tasks reached new heights (Devlin et al., 2018; Liu et al., 2019). Un- fortunately, these models use an enormous amount of parameters. Inference on resource-limited hard- ware such as edge-devices is thus impractical. A solution to reduce the computational burden of these networks is to lower numerical precision. Consequently, numerical values can be represented using fewer bits (Tang and Kwan, 1993; March- esi et al., 1993). This method called quantization has the advantage of providing good compression rates with minimal loss in accuracy. It is also con- veniently supported by most hardware. Properly quantizing the Transformer would allow computa- tional speed gains at inference, as well as deploy- ment on more constrained devices. In this work, we propose a quantization-aware training strategy for the entire Transformer architec- ture. Our method is easy to implement and results are consistent with the full-precision Transformer. We test our approach on multiple translation tasks such as WMT14 EN-FR and WMT14 EN-DE and obtain state-of-the-art quantization results. In com- parison with full-precision, our quantized models score equal or higher BLEU on most tasks. We are, to the best of our knowledge, the first to show that the Transformer architecture can be fully quantized without impairing translation quality. We also per- form an ablation study and show that quantizing specific …

Content truncated to 500 words