Bilingual Subword Segmentation for Neural Machine Translation

Proceedings of the 28th International Conference on Computational Linguistics, pages 4287–4297 Barcelona, Spain (Online), December 8-13, 2020 4287 Bilingual Subword Segmentation for Neural Machine Translation Hiroyuki Deguchi1, Masao Utiyama2, Akihiro Tamura3, Takashi Ninomiya1, Eiichiro Sumita2 1Ehime University 2National Institute of Information and Communications Technology 3Doshisha University 1{deguchi@ai., ninomiya@}cs.ehime-u.ac.jp 2{mutiyama, eiichiro.sumita}@nict.go.jp 3aktamura@mail.doshisha.ac.jp Abstract This paper proposed a new subword segmentation method for neural machine translation, “Bilin- gual Subword Segmentation,” which tokenizes sentences to minimize the difference between the number of subword units in a sentence and that of its translation. While existing subword seg- mentation methods tokenize a sentence without considering its translation, the proposed method tokenizes a sentence by using subword units induced from bilingual sentences; this method could be more favorable to machine translation. Evaluations on WAT Asian Scientific Paper Excerpt Corpus (ASPEC) English-to-Japanese and Japanese-to-English translation tasks and WMT14 English-to-German and German-to-English translation tasks show that our bilingual subword segmentation improves the performance of Transformer neural machine translation (up to +0.81 BLEU). 1 Introduction Subword units have recently been widely used in neural machine translation (NMT) to solve open vocab- ulary problems. Byte Pair Encoding (BPE) (Sennrich et al., 2016) is a dominant subword segmentation method for NMT, but it is designed for segmented languages in which words are divided by spaces. Kudo (2018) has proposed a subword segmentation method based on a unigram language model, that can be applied to non-segmented languages such as Chinese and Japanese. Both BPE and the unigram language model tokenize sentences by minimizing the number of segments under a limitation on sub- word vocabulary size, which relies on a data compression principle. In these existing segmentations, a sentence is segmented without considering its translation, and therefore the segmented sentence might not be optimal for NMT. This paper proposes a new subword segmentation method for NMT, “Bilingual Subword Segmenta- tion,” which tokenizes sentences by using subword units induced from bilingual sentences. The proposed method is based on a unigram language model like Kudo (2018) because we aim to improve translation performance for non-segmented languages1. In particular, the proposed segmentation tokenizes bilin- gual sentences (i.e., training data for NMT) by selecting subword sequence pairs with similar numbers of segments, from segmentation candidates of the source and target language sentences obtained by a unigram language model. For segmentation of monolingual source language sentences (i.e., test data for NMT), an LSTM-based subword segmenter for the source language is preliminarily learned from the source side of segmented bilingual sentences, and monolingual source language sentences are tokenized by the learned subword segmenter. Our bilingual segmentation encourages one-to-one mappings between segments across languages be- cause it minimizes the difference between the number of segments in a sentence and that of its translation. As a result, subword units segmented by our bilingual segmentation can be expected to be more helpful for NMT than conventional subword units. For example, consider the situation in which two Japanese This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// …

Content truncated to 500 words