Nonparametric Word Segmentation for Machine Translation

Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 815–823, Beijing, August 2010 Nonparametric Word Segmentation for Machine Translation ThuyLinh Nguyen Stephan Vogel Noah A. Smith Language Technologies Institute Carnegie Mellon University {thuylinh,vogel,nasmith}@cs.cmu.edu Abstract We present an unsupervised word seg- mentation model for machine translation. The model uses existing monolingual seg- mentation techniques and models the joint distribution over source sentence segmen- tations and alignments to the target sen- tence. During inference, the monolin- gual segmentation model and the bilin- gual word alignment model are coupled so that the alignments to the target sen- tence guide the segmentation of the source sentence. The experiments show improve- ments on Arabic-English and Chinese- English translation tasks. 1 Introduction In statistical machine translation, the smallest unit is usually the word, defined as a token delimited by spaces. Given a parallel corpus of source and target text, the training procedure first builds a word alignment, then extracts phrase pairs from this word alignment. However, in some languages (e.g., Chinese) there are no spaces between words. The same problem arises when translating be- tween two very different languages, such as from a language with rich morphology like Hungarian or Arabic to a language with poor morphology like English or Chinese. A single word in a mor- phologically rich language is often the composi- tion of several morphemes, which correspond to separate words in English.1 1We will use the terms word segmentation, morphologi- cal analysis, and tokenization more or less interchangeably. Often some preprocessing is applied involving word segmentation or morphological analysis of the source and/or target text. Such preprocess- ing tokenizes the text into morphemes or words, which linguists consider the smallest meaning- bearing units of the language. Take as an ex- ample the Arabic word “fktbwha” and its En- glish translation “so they wrote it”. The preferred segmentation of “fktbwha” would be “f-ktb-w-ha (so-wrote-they-it),” which would allow for a one- to-one mapping between tokens in the two lan- guages. However, the translation of the phrase in Hebrew is “wktbw ath”. Now the best segmen- tation of the Arabic words would be “fktbw-ha,” corresponding to the two Hebrew words. This ex- ample shows that there may not be one correct segmentation that can be established in a prepro- cessing step. Rather, tokenization depends on the language we want to translate into and needs to be tied in with the alignment process. In short, we want to find the tokenization yielding the best alignment, and thereby the best translation sys- tem. We propose an unsupervised tokenization method for machine translation by formulating a generative Bayesian model to “explain” the bilin- gual training data. Generation of a sentence pair is described as follows: first a monolingual to- kenization model generates the source sentence, then the alignment model generates the target sen- tence through the alignments with the source sen- tence. Breaking this generation process into two steps provides flexibility to incorporate existing monolingual morphological segmentation mod- els such as those of Mochihashi et al. …

Content truncated to 500 words