Chunk-based Nearest Neighbor Machine Translation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4228 - 4245 December 7-11, 2022 ©2022 Association for Computational Linguistics Chunk-based Nearest Neighbor Machine Translation Pedro Henrique Martins�� Zita Marinho�� André F. T. Martins��� �Instituto de Telecomunicações �Unbabel �Institute of Systems and Robotics �DeepMind �LUMLIS (Lisbon ELLIS Unit), Instituto Superior Técnico Lisbon, Portugal pedrohenriqueamartins@tecnico.ulisboa.pt, zmarinho@google.com, andre.t.martins@tecnico.ulisboa.pt. Abstract Semi-parametric models, which augment gen- eration with retrieval, have led to impressive re- sults in language modeling and machine transla- tion, due to their ability to retrieve fine-grained information from a datastore of examples. One of the most prominent approaches, kNN-MT, exhibits strong domain adaptation capabilities by retrieving tokens from domain-specific data- stores (Khandelwal et al., 2021). However, kNN-MT requires an expensive retrieval opera- tion for every single generated token, leading to a very low decoding speed (around 8 times slower than a parametric model). In this paper, we introduce a chunk-based kNN-MT model which retrieves chunks of tokens from the data- store, instead of a single token. We propose several strategies for incorporating the retrieved chunks into the generation process, and for se- lecting the steps at which the model needs to search for neighbors in the datastore. Experi- ments on machine translation in two settings, static and “on-the-fly” domain adaptation, show that the chunk-based kNN-MT model leads to significant speed-ups (up to 4 times) with only a small drop in translation quality.1 1 Introduction Machine translation has seen remarkable ad- vances due to increasingly powerful neural mod- els (Sutskever et al., 2014; Bahdanau et al., 2015; Vaswani et al., 2017). Most deployed systems are fully-parametric (the training data is fully com- pressed into the parameters of a neural model), but they often struggle when translating rare words or out-of-domain sentences (Koehn and Knowles, 2017), commonly requiring several stages of fine- tuning to adapt to data drift or to new domains. Recently, semi-parametric methods have shown great promise, by combining the strengths of para- metric models with external databases of parallel 1The code is available at https://github.com/ deep-spin/chunk-based_knn-mt. sentences, such as translation memories (Gu et al., 2018; Zhang et al., 2018; Bapna and Firat, 2019a; Meng et al., 2021; Zheng et al., 2021a; Jiang et al., 2021; Martins et al., 2022). One of the most prominent semi-parametric mod- els for machine translation is the k-Nearest Neigh- bor Machine Translation model (kNN-MT) (Khan- delwal et al., 2021), which has led to impressive results, particularly in domain adaptation settings, without requiring fine-tuning. The kNN-MT model constructs domain-specific datastores of parallel sentences and, at inference time, it retrieves similar examples from these datastores, which are used to improve the generation process, through the in- terpolation of probability distributions. However, kNN-MT only retrieves single tokens—this is inef- ficient, since the model needs to consult the datas- tore at every generation step, an expensive opera- tion. Consequently, its decoding speed is around 8 times slower than that of a fully-parametric model. Recent work has introduced several techniques to speed up kNN-MT. Meng et …

Content truncated to 500 words