Low-resource Neural Machine Translation with Cross-modal Alignment

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10134 - 10146 December 7-11, 2022 ©2022 Association for Computational Linguistics Low-resource Neural Machine Translation with Cross-modal Alignment Zhe Yang1,2, Qingkai Fang1,2, Yang Feng1,2∗ 1 Key Laboratory of Intelligent Information Processing Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS) 2 University of Chinese Academy of Sciences, Beijing, China {yangzhe22s1,fangqingkai21b,fengyang}@ict.ac.cn Abstract How to achieve neural machine translation with limited parallel data? Existing techniques of- ten rely on large-scale monolingual corpora, which is impractical for some low-resource languages. In this paper, we turn to connect several low-resource languages to a particular high-resource one by additional visual modal- ity. Specifically, we propose a cross-modal contrastive learning method to learn a shared space for all languages, where both a coarse- grained sentence-level objective and a fine- grained token-level one are introduced. Ex- perimental results and further analysis show that our method can effectively learn the cross- modal and cross-lingual alignment with a small amount of image-text pairs and achieves sig- nificant improvements over the text-only base- line under both zero-shot and few-shot sce- narios. Our code could be found at https: //github.com/ictnlp/LNMT-CA. 1 Introduction Neural machine translation (NMT) has shown ex- cellent performance and becomes the dominant paradigm of machine translation. However, NMT is a data-driven approach, which requires a large amount of parallel data. When the data is insuffi- cient, it is impractical to train a reasonable NMT model. Unfortunately, there are many languages in the world for which sufficient training data is not available, and sometimes there is no parallel data at all. Therefore, the translation of low-resource languages is a vital challenge for NMT. In recent years, researchers have attempted to improve the performance of NMT for low-resource languages. Lample et al. (2018a) proposed an unsu- pervised approach to learn weak mappings between languages with large amount of monolingual data (>1M), which is also costly for low-resource lan- guages. Liu et al. (2020); Lin et al. (2020b); Pan ∗Corresponding author: Yang Feng. <EN> a dog is running in the snow <DE> ein hund rennt im schnee <FR> un chien court dans la neige zero-shot few-shot MT Cross-modal Alignment <DE> <FR> Figure 1: We aim at realizing zero-shot and few-shot machine translation for the low-resource language. Dif- ferent languages with the same meanings are projected to a shared space by cross-modal alignment. et al. (2021); Gu and Feng (2022) proposed multi- lingual NMT models, which learn a shared space of multiple languages to achieve translations between languages that appear in the training set but do not have the corresponding parallel data. However, they still require auxiliary parallel data of source and target languages along with many other lan- guages, which is still infeasible for low-resource languages. In recent years, with increasing attention of multi-modal tasks, resource of image-text pairs have become more abundant. Inspired by re- cent efforts on cross-modal alignment (Radford et al., 2021; Li et al., 2021; Fang et al., 2022), in this paper, we propose …

Content truncated to 500 words