Target-Side Augmentation for Document-Level Machine Translation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 65–77 July 9-14, 2023 ©2023 Association for Computational Linguistics Binary and Ternary Natural Language Generation Zechun Liu∗ Reality Labs, Meta Inc. zechunliu@meta.com Barlas O˘guz∗ Meta AI barlaso@meta.com Aasish Pappu Meta AI aasish@fb.com Yangyang Shi Reality Labs, Meta Inc. yyshi@meta.com Raghuraman Krishnamoorthi Reality Labs, Meta Inc. raghuraman@meta.com Abstract Ternary and binary neural networks enable multiplication-free computation and promise multiple orders of magnitude efficiency gains over full-precision networks if implemented on specialized hardware. However, since both the parameter and the output space are highly discretized, such networks have proven very difficult to optimize. The difficulties are compounded for the class of transformer text generation models due to the sensitivity of the attention operation to quantization and the noise-compounding effects of autoregressive decoding in the high-cardinality output space. We approach the problem with a mix of statistics-based quantization for the weights and elastic quantization of the activations and demonstrate the first ternary and binary transformer models on the downstream tasks of summarization and machine translation. Our ternary BART base achieves an R1 score of 41 on the CNN/DailyMail benchmark, which is merely 3.9 points behind the full model while being 16x more efficient. Our binary model, while less accurate, achieves a highly non- trivial score of 35.6. For machine translation, we achieved BLEU scores of 21.7 and 17.6 on the WMT16 En-Ro benchmark, compared with a full precision mBART model score of 26.8. We also compare our approach in the 8-bit activation setting, where our ternary and even binary weight models can match or outperform the best existing 8-bit weight models in the literature. Our code and models are available at: https://github.com/facebookresearch/ Ternary_Binary_Transformer. 1 Introduction Generative pre-trained transformers (Brown et al., 2020; Lewis et al., 2020; Radford et al., 2018) have emerged as powerful and generic tools, driving breakthroughs not only in language understanding but the field of AI in general. These models owe ∗Equal contribution their success mainly to their seemingly infinite abil- ity to scale to ever-larger data and model sizes. Un- fortunately, such scaling comes at the cost of large computational requirements, putting extensively large generative transformers out of reach of all but the most resource-rich institutions. Even moder- ately sized pre-trained transformers have limited applications due to their size and computational cost. Making generative transformers more effi- cient is imperative for widening their use to more devices and practical applications. In this work, we explore making generative pre- trained transformers more efficient via the quanti- zation of their weights and activations. Quantizing the weights of a neural network is useful for com- pression and allows the model to be stored more ef- ficiently. However, compression alone does not re- duce computation costs since the network’s activa- tions need to be computed in full precision. Quan- tizing both weights and activations allows compu- tation to be performed with lower precision, po- tentially leading to significant efficiency gains de- pending on the quantization level …

Content truncated to 500 words