Neural Machine Translation from Historical Japanese to Contemporary Japanese Using Diachronically Domain-Adapted Word Embeddings

Neural Machine Translation from Historical Japanese to Contemporary Japanese Using Diachronically Domain-Adapted Word Embeddings Masashi Takaku Tosho Hirasawa Mamoru Komachi Kanako Komiya Ibaraki University 4-12-1, Nakanarusawa, Hitachi, Ibaraki, 316-8511, Japan {16t4027n, kanako.komiya.nlp}@vc.ibaraki.ac.jp Tokyo Metropolitan University 6-6 Asahigaoka, Hino, Tokyo, 191-0065, Japan {hirasawa-tosho@ed., komachi@}tmu.ac.jp Abstract This paper describes the first trial of neu- ral machine translation (NMT) from historical Japanese to contemporary Japanese. To com- pensate for the lack of parallel data, we used pre-trained word embeddings for the input of the system and performed diachronic domain adaptation in the order of time. We investi- gated and compared an NMT system without pre-trained word embeddings, an NMT sys- tem with pre-trained word embeddings trained with contemporary Japanese, an NMT sys- tem with word embeddings diachronically domain-adapted at one time, and NMT sys- tems with word embeddings that were grad- ually domain-adapted in the order of time. Al- though our system did not outperform statisti- cal machine translation, experiments revealed that diachronic domain adaptation is effective, especially if it is performed in the order of time. 1 Introduction This paper describes a neural machine transla- tion (NMT) system for translation from histori- cal Japanese to contemporary Japanese. In recent years, machine translation using deep learning, or NMT, has been intensively studied. Although there is a study of statistical machine translation (SMT) from historical Japanese to contemporary Japanese (Hoshino et al., 2014), to the best of our knowl- edge, there have been no studies on an NMT sys- tem that translates historical Japanese to contempo- rary Japanese. NMT systems generally output flu- ent translations. Therefore, NMT is expected to im- prove the fluency of contemporary Japanese transla- tion. However, it is difficult to obtain a model with high performance when only small parallel corpora are available, as NMT systems usually require large parallel corpora for training (Koehn and Knowles, 2017). Because the available parallel corpus of his- torical and contemporary Japanese is small, NMT is not appropriate for translation from historical Japanese to contemporary Japanese. To improve translation performance, translation models are sometimes initialized with pre-trained word embeddings trained with a large corpus for lan- guage pairs that do not have sufficient parallel cor- pora (Qi et al., 2018). We believe that this method can also be effective for the translation model from historical Japanese to contemporary Japanese. To obtain high-quality word embeddings, it is de- sirable to train them using a large training corpus. Therefore, the use of word embeddings trained with a contemporary Japanese corpus that is much larger than the historical Japanese corpus is expected. However, when the word embeddings trained with contemporary Japanese are directly used for the translation model, the model is expected to have poorer performance because the domains of the word embeddings and inputs differ from each other. In addition, because the parallel corpus of his- torical and contemporary Japanese contains litera- ture from different time periods, the meaning of the words or the words themselves may change accord- ing to the period. We thus propose …

Content truncated to 500 words