Large Language Models Are State-of-the-Art Evaluators of Translation Quality

Towards Efficient Universal Neural Machine Translation Biao Zhang∗ School of Informatics University of Edinburgh b.zhang@ed.ac.uk Humans benefit from communication but suffer from language barriers. Machine translation (MT) aims to overcome such barriers by automatically transforming information from one language to an- other. With the rapid development of deep neu- ral networks, neural machine translation (NMT) – especially Transformer (Vaswani et al., 2017) – has achieved great success in recent years, de- livering state-of-the-art and even near human per- formance on many bilingual text-based translation tasks (Akhbardeh et al., 2021). However, chal- lenges remain particularly in 1) efficiency where a massive NMT model is a computational bottle- neck for training and decoding, and 2) universality where extending NMT beyond bilingual and text- based scenarios (such as multilingual and speech- to-text translation) is still non-trivial. In this the- sis, we investigate ways of developing simple and effective neural architectures to address these two challenges. NMT is resource-hungry. Achieving high- quality translation demands complex network ar- chitectures and a large number of model parame- ters, which often takes hundreds or even thousands of training GPU hours and leads to slow inference. We tackle this computational inefficiency issue via three aspects: 1) simplifying model architectures, where we propose a lightweight recurrent network and root mean square layer normalization to enable higher model parallelization, as well as a merged attention network paired with depth-scaled initial- ization to improve deep Transformer; 2) explor- ing representation redundancy, where we demon- strate the feasibility of sparsifying encoder outputs ∗Now at Google Deepmind. ∗© 2023 The authors. This article is licensed under a Creative Commons 4.0 licence, no derivative works, attribution, CC- BY-ND. in Transformer and propose a rectified linear atten- tion to induce sparse attention weights efficiently; and 3) semi-autoregressive modeling, where we re- lax the independence assumption by allowing gen- eration from the left-to-right and right-to-left di- rections simultaneously. Apart from benefiting ef- ficiency, these techniques also lay the foundation for our research on universality, another topic of this thesis. MT should be universal, i.e., being capable of transforming information between any languages in any modalities. Unfortunately, NMT still struggles with poor language coverage and cross- modality gap. As a step towards universal MT, we focus on (massively) multilingual NMT and direct speech-to-text translation (ST). Multilingual NMT suffers from capacity bottleneck and off-target translation; we thus study methods of increasing modeling capacity for multilingual Transformer, and propose random online backtranslation to bridge zero-short language pairs. We further ex- plore when and where language-specific model- ing matters via conditional language-specific rout- ing, discovering the trade-off between shared and language-specific capacity. Unlike textual NMT, the modality gap between speech and text hinders ST. We narrow this gap by inventing adaptive fea- ture selection, which automatically filters out un- informative speech features, improving translation as well as inference speed. Next, we extend our study to document-level speech translation to ad- dress the question whether and how context helps ST. We adopt contextual modeling for ST, and show its effectiveness on …

Content truncated to 500 words