Synthetic Data for Neural Machine Translation of Spoken-Dialects

Synthetic Data for Neural Machine Translation of Spoken-Dialects Hany Hassan, Mostafa Elaraby, Ahmed Y. Tawfik Microsoft AI & Research hanyh,a-moelar,atawfik@microsoft.com Abstract In this paper, we introduce a novel approach to gener- ate synthetic data for training Neural Machine Translation systems. The proposed approach supports language vari- ants and dialects with very limited parallel training data. This is achieved using a seed data to project words from a closely-related resource-rich language to an under-resourced language variant via word embedding representations. The proposed approach is based on localized embedding projec- tion of distributed representations which utilizes monolin- gual embeddings and approximate nearest neighbors queries to transform parallel data across language variants. Our approach is language independent and can be used to generate data for any variant of the source language such as slang or spoken dialect or even for a different language that is related to the source language. We report experimental results on Levantine to English translation using Neural Ma- chine Translation. We show that the synthetic data can pro- vide significant improvements over a very large scale system by more than 2.8 Bleu points and it can be used to provide a reliable translation system for a spoken dialect which does not have sufficient parallel data. 1. Introduction Neural Machine Translation (NMT) [1] has achieved state- of-the-art translation quality in various research evaluations campaigns [29] and online large scale production systems [3] and [4]. With such large systems, NMT showed that it can scale up to huge amounts of parallel data. However, such large parallel data is not widely available for all domains and language styles. Usually parallel training data is widely available in written formal languages such as UN and Eu- roparl data. Real-time speech translation systems support sponta- neous, open-domain conversations between speakers of dif- ferent languages. Speech Translation Systems are becoming a practical tool that can help in eliminating language barriers for spoken languages. Those machine translation systems are usually trained using NMT with large amount of parallel data adapted from written data to the spoken style [5]. This is a valid approach when the spoken and written languages are similar and mainly differ in style. For many languages, the written and spoken forms are quite different §2. While the written form usually has an abundance of parallel data avail- able to train a reliable NMT system; the spoken form may not have any parallel data or even, in some cases, a standardized written form. In this paper, we propose a novel approach to generate synthetic data for NMT. The proposed approach transforms a given parallel corpus between a written language and a tar- get language to a parallel corpus between the spoken dialect variant and the target language. Our approach is language independent and can be used to generate data for any variant of the source language such as slang, spoken dialect or social media style or even for a different language that is closely- related to such source language. The synthetic data generation approach is …

Content truncated to 500 words