Pre-training Synthetic Cross-lingual Decoder for Multilingual Samples Adaptation in E-Commerce Neural Machine Translation

Pre-training Synthetic Cross-lingual Decoder for Multilingual Samples Adaptation in E-Commerce Neural Machine Translation Kamal Kumar Gupta, Soumya Chennabasavaraj,† Nikesh Garera,† and Asif Ekbal Department of Computer Science and Engineering, Indian Institute of Technology Patna, Patna, India †Flipkart, India kamal.pcs17, asif@iitp.ac.in †soumya.cb, nikesh.garera@flipkart.com Abstract Availability of the user reviews in vernacu- lar languages is helpful for the users to get information regarding the products. Since most of the e-commerce websites allow the reviews in English language only, it is important to provide the translated ver- sions of the reviews to the non-English speaking users. Translation of the user reviews from English to vernacular lan- guages is a challenging task, predomi- nantly due to the lack of sufficient in- domain datasets. In this paper, we present a pre-training technique which is used to adapt and improve the single multilingual neural machine translation (NMT) model for the low-resource language pairs. The pre-trained model contains a special syn- thetic cross-lingual decoder trained over the cross-lingual target samples where the phrases are replaced with their translated counterparts. After pre-training, the model is adapted to multiple samples of the low- resource language pairs using incremen- tal learning. We perform the experiments over eight low-resource and three high re- source language pairs from the generic and product review domains. Through our proposed pre-training, we achieve upto 4.35 BLEU improvements compared to the baseline and 2.13 BLEU points compared to the previous code-switched pre-trained models. The review domain outputs are evaluated in human evaluators in the e- commerce company Flipkart. © 2022 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. 1 Introduction Neural machine translation models (Bahdanau et al., 2015; Vaswani et al., 2017) are effective for a specific language pair or domain when trained on a large amount of parallel corpus. It is often dif- ficult to obtain such a large corpus, especially in non-English languages and in specialized domains such as product reviews (Gupta et al., 2021). Cur- rently, in the e-commerce domain, providing the translation of the user reviews in vernacular lan- guages is a need. In a multilingual country like India where English is not a primary language, re- views in local languages will be very helpful for the users as well as e-commerce platforms like Flipkart. As of December 2021, Flipkart leads1 in the Indian e-commerce market with a market share of 31.9%. In the process of building a one- to-many multilingual translation system to trans- late the low-resource review domain data on the e- commerce platform Flipkart from English to mul- tiple Indian languages, we propose a synthetic de- coder based pre-training approach. To see the im- pact of the proposed model on translation quality, we perform experiments over the general domain data available publicly. Along with it, we also evaluate our model for review domain data using English-Hindi, English-French and English-Tamil testset. Recently, pre-training based NMT (Lewis et al., 2019; Devlin et al., 2019) models have attracted at- tention to improve …

Content truncated to 500 words