Evaluating Machine Translation in Cross-lingual E-Commerce Search

Domain-Specific Text Generation for Machine Translation Yasmin Moslem yasmin.moslem@adaptcentre.ie Andy Way andy.way@adaptcentre.ie School of Computing, Dublin City University, Dublin, Ireland Rejwanul Haque rejwanul.haque@adaptcentre.ie School of Computing, National College of Ireland, Dublin, Ireland John D. Kelleher john.kelleher@adaptcentre.ie Technological University Dublin, Dublin, Ireland Abstract Preservation of domain knowledge from the source to target is crucial in any translation workflow. It is common in the translation industry to receive highly specialized projects, where there is hardly any parallel in-domain data. In such scenarios where there is insufficient in-domain data to fine-tune Machine Translation (MT) models, producing translations that are consistent with the relevant context is challenging. In this work, we propose a novel approach to domain adaptation leveraging state-of-the-art pretrained language models (LMs) for domain-specific data augmentation for MT, simulating the domain characteristics of either (a) a small bilingual dataset, or (b) the monolingual source text to be translated. Combining this idea with back-translation, we can generate huge amounts of synthetic bilingual in-domain data for both use cases. For our investigation, we use the state-of-the-art Transformer architecture. We employ mixed fine-tuning to train models that significantly improve translation of in-domain texts. More specifically, in both scenarios, our proposed methods achieve improvements of approximately 5-6 BLEU and 2-3 BLEU, respectively, on the Arabic-to-English and English-to-Arabic language pairs. Furthermore, the outcome of human evaluation corroborates the automatic evaluation results. 1 Introduction Neural Machine Translation (NMT) has the ability to produce good quality translations in terms of both fluency and adequacy (Bahdanau et al., 2015). Nevertheless, NMT still faces some chal- lenges when it comes to translation of out-of-domain texts (Koehn and Knowles, 2017). Domain adaptation of MT systems on in-domain parallel texts has been an active area of research to han- dle this situation. Among popular contributions to the domain adaptation research, Luong and Manning (2015) proposed to adapt an already existing NMT system to a new domain, with further training on the in-domain data only. In an effort to avoid overfitting on the in-domain data, Chu et al. (2017) employed the mixed fine-tuning approach, resuming training the base- line NMT model on a mix of in-domain and out-of-domain data. Other researchers suggested adding domain tags to either the source or target sentences of the in-domain data, to inform the NMT model about the domain during training and decoding (Britz et al., 2017; Kobus et al., 2017; Stergiadis et al., 2021). In this sense, several research works on domain adaptation assume the availability of in- domain data. However, in-domain data scarcity is common in translation settings, due to the lack of specialized datasets and terminology, or inconsistency and inaccuracy of available in- domain translations. To tackle this problem, researchers have proposed diverse approaches, such as utilizing large monolingual datasets through selecting instances related to a given test Proceedings of the 15th Biennial Conference of the Association for Machine Translation in the Americas Orlando, USA, September 12-16, 2022. Volume 1: Research Track 14 set, then automatically translating this source-synthetic corpus, and finally fine-tuning the gen- eral …

Content truncated to 500 words