Neural Machine Translation of Logographic Language Using Sub-character Level Information

Proceedings of the Third Conference on Machine Translation (WMT), Volume 1: Research Papers, pages 17–25 Belgium, Brussels, October 31 - Novermber 1, 2018. c⃝2018 Association for Computational Linguistics https://doi.org/10.18653/v1/W18-64003 Neural Machine Translation of Logographic Languages Using Sub-character Level Information Longtu Zhang Tokyo Metropolitan University 6-6 Asahigaoka, Hino, Tokyo 191-0065, Japan zhang-longtu@ed.tmu.ac.jp Mamoru Komachi Tokyo Metropolitan University 6-6 Asahigaoka, Hino, Tokyo 191-0065, Japan komachi@tmu.ac.jp Abstract Recent neural machine translation (NMT) systems have been greatly improved by encoder-decoder models with attention mech- anisms and sub-word units. However, im- portant differences between languages with logographic and alphabetic writing systems have long been overlooked. This study fo- cuses on these differences and uses a simple approach to improve the performance of NMT systems utilizing decomposed sub-character level information for logographic languages. Our results indicate that our approach not only improves the translation capabilities of NMT systems between Chinese and English, but also further improves NMT systems be- tween Chinese and Japanese, because it uti- lizes the shared information brought by simi- lar sub-character units. 1 Introduction Neural machine translation (Cho et al., 2014) (NMT) systems based on sequence-to-sequence models (Sutskever et al., 2014) have recently be- come the de facto standard architecture. The models use attention mechanisms (Bahdanau et al., 2015; Luong et al., 2015) to keep records of all encoding results, and can focus on particu- lar parts of these results during decoding, so that the model can produce longer and more accurate translations. Sub-word units are another tech- nique first introduced by Sennrich’s (2016) appli- cation of the byte pair encoding (BPE) algorithm, and are used to break up words in both source and target sentences into sequences of smaller units, learned without supervision. This alleviates the risk of producing <unk> symbols when the model encounters infrequent “unknown” words, also known as the out-of-vocabulary (OOV) problem. Moreover, sub-word units, which can be viewed as learned stems and affixes, can help the NMT model better encode the source sentence and de- code the target sentence, particularly when the source and target languages share some similar- ities. Almost all of the methods used to improve NMT systems were developed for alphabetic lan- guages such as English, French, and German as either the source or target language, or both. An alphabetic language typically uses an alphabet: a small set of letters (basic writing symbols) that each roughly represents a phoneme in the spo- ken language. Words are composed by ordered letters, and sentences are composed by space- segmented ordered words. However, in other major writing systems—namely, logographic (or character-based) languages such as Chinese, Japanese, and traditional Korean—strokes are used to construct ideographs; ideographs are used to construct characters, which are the basic units for meaningful words. Words can then further compose sentences. In alphabetic languages, sub-word units are easy to identify, whereas in logographic languages, a similar effect can be achieved only if sub-character level information is taken into consideration.1 Having noticed this significant difference between these two writing systems, Shi et al. (2015), Liu …

Content truncated to 500 words