Domain Informed Neural Machine Translation: Developing Translation Services for Healthcare Enterprise

Domain Informed Neural Machine Translation: Developing Translation Services for Healthcare Enterprise Sahil Manchanda and Galina Grunin Optum {sahil_manchanda, galina.grunin}@optum.com Abstract Neural Machine Translation (NMT) is a deep learning based approach that has achieved outstanding results lately in the translation community. The performance of NMT systems, however, is dependent on the availability of large amounts of in- domain parallel corpora. The business enterprises in domains such as legal and healthcare require specialized vocabulary but translation systems trained for a gen- eral purpose do not cater to these needs. The data in these domains is either hard to acquire or is very small in comparison to public data sets. This is a detailed report of using an open- source library to implement a machine translation system and successfully cus- tomizing it for the needs of a particular client in the healthcare domain. This report details the chronological development of every component of this system, namely, extraction of data from in-domain health- care documents, a pre-processing pipeline for the data, data alignment and augmenta- tion, training and a fully automated and ro- bust deployment pipeline. This work pro- poses an efficient way for the continuous deployment of newly trained deep learning models. The deployed translation models are optimized for both inference time and cost. c⃝ 2020 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. 1 Introduction The emergence of Neural Machine Translation (NMT) was sparked by the use of Recurrent Neu- ral Networks (RNN) for machine translation. The RNN encoder in this approach is responsible for encoding the source language phrase into a fixed- length vector. This vector is then decoded into the target language (Cho et. al., 2014). Some approaches also use Long Short Term Memory (LSTM) (Hochreiter, 1997) for this task (Sutskever et al., 2014). Sequence to Sequence models with attention (Bah- danau et al., 2014) started coming to fruition with the advent of the idea that the LSTM layers stacked on top of each other stopped improving at about a depth of 4. The attention-based models were es- sentially of two types, local and global. The at- tention models proved to be much more efficient in translating long sentences as compared to non- attention models (Luong et al., 2015). But long sentences still exhibited ”exposure bias” which led to the emergence of attention models that attend over the input and generated outputs separately (Paulus, Xiong, Socher, 2017). Despite these remarkable advances in NMT, one of the major problems still faced by some enterprises is domain-specific translation where the general- purpose translators do not perform well. Leverag- ing in-domain corpora to skew a general purpose translator successfully has been an area of inter- est in NMT in recent years. One of the successful techniques to do this has been mixed fine-tuning (Chu, Chenhui and Wang, Rui, 2018) which sug- gests to train an NMT model on out-of-domain corpora until model convergence and then resume training from step 1 on a mix …

Content truncated to 500 words