Efficient Cluster-Based k-Nearest-Neighbor Machine Translation

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2175 - 2187 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Efficient Cluster-Based k-Nearest-Neighbor Machine Translation Dexin Wang∗ Tianjin University dexinwang@tju.edu.cn Kai Fan* Alibaba DAMO Academy k.fan@alibaba-inc.com Boxing Chen Alibaba DAMO Academy boxing.cbx@alibaba-inc.com Deyi Xiong† Tianjin University dyxiong@tju.edu.cn Abstract k-Nearest-Neighbor Machine Translation (kNN-MT) has been recently proposed as a non-parametric solution for domain adaptation in neural machine translation (NMT). It aims to alleviate the performance degradation of advanced MT systems in translating out-of- domain sentences by coordinating with an additional token-level feature-based retrieval module constructed from in-domain data. Pre- vious studies (Khandelwal et al., 2021; Zheng et al., 2021a) have already demonstrated that non-parametric NMT is even superior to models fine-tuned on out-of-domain data. In spite of this success, kNN retrieval is at the expense of high latency, in particular for large datastores. To make it practical, in this paper, we explore a more efficient kNN-MT and propose to use clustering to improve the retrieval efficiency. Concretely, we first propose a cluster-based Compact Network for feature reduction in a contrastive learning manner to compress context features into 90+% lower dimensional vectors. We then suggest a cluster-based pruning solution to filter out 10%~40% redundant nodes in large datastores while retaining translation quality. Our proposed methods achieve better or comparable performance while reducing up to 57% inference latency against the advanced non-parametric MT model on several ma- chine translation benchmarks. Experimental results indicate that the proposed methods maintain the most useful information of the original datastore and the Compact Net- work shows good generalization on unseen domains. Codes are available at https: //github.com/tjunlp-lab/PCKMT. 1 Introduction Recently, non-parametric approaches (Khandelwal et al., 2021; Zheng et al., 2021a,b; Jiang et al., 2021) have been successfully applied to neural ∗ Equal contribution. † Corresponding author. Model speed (token/s) BLEU MT 913.48 37.50 AK-MT (k=4) 642.43 46.32 Table 1: The inference speed comparison on the same IT-domain test set. AK-MT denotes the adaptive kNN- MT. machine translation (NMT) for domain adaptation with retrieval pipelines. Given an advanced MT model, they generally involve two steps: • It builds a cached memory, usually called datastore, in advance by extracting the con- text representations of the penultimate layer of the given NMT model corresponding to each target token from in-domain data. • At inference, it retrieves the k nearest neigh- bors of the context representation for each generated token from the constructed datas- tore and then integrates external kNN transla- tion probabilities derived from these retrievals to adjust the translation. The accessibility of any provided datastore dur- ing translation makes them interpretable. Mean- while, the reliability of these approaches gives the credit to the datastore quality. In spite of signif- icant translation improvements, analyses on the datastore behavior have not been fully explored yet. We empirically observe that the construction of datastore is not optimal for retrieval from two aspects: retrieval latency and semantic distribution. Retrieval Latency. As shown in Table 1, …

Content truncated to 500 words