Large Scale Decipherment for Out-of-Domain Machine Translation

Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 266–275, Jeju Island, Korea, 12–14 July 2012. c⃝2012 Association for Computational Linguistics Large Scale Decipherment for Out-of-Domain Machine Translation Qing Dou and Kevin Knight Information Sciences Institute Department of Computer Science University of Southern California {qdou,knight}@isi.edu Abstract We apply slice sampling to Bayesian de- cipherment and use our new decipherment framework to improve out-of-domain machine translation. Compared with the state of the art algorithm, our approach is highly scalable and produces better results, which allows us to decipher ciphertext with billions of tokens and hundreds of thousands of word types with high accuracy. We decipher a large amount of monolingual data to improve out-of-domain translation and achieve significant gains of up to 3.8 BLEU points. 1 Introduction Nowadays, state of the art statistical machine trans- lation (SMT) systems are built using large amounts of bilingual parallel corpora. Those corpora are used to estimate probabilities of word-to-word trans- lation, word sequences rearrangement, and even syntactic transformation. Unfortunately, as paral- lel corpora are expensive and not available for ev- ery domain, performance of SMT systems drops significantly when translating out-of-domain texts (Callison-Burch et al., 2008). In general, it is easier to obtain in-domain mono- lingual corpora. Is it possible to use domain specific monolingual data to improve an MT system trained on parallel texts from a different domain? Some re- searchers have attempted to do this by adding a do- main specific dictionary (Wu et al., 2008), or mining unseen words (Daum´e and Jagarlamudi, 2011) us- ing one of several translation lexicon induction tech- niques (Haghighi et al., 2008; Koehn and Knight, 2002; Rapp, 1995). However, a dictionary is not al- ways available, and it is difficult to assign probabil- ities to a translation lexicon. (Ravi and Knight, 2011b) have shown that one can use decipherment to learn a full translation model from non-parallel data. Their approach is able to find translations, and assign probabilities to them. But their work also has certain limitations. First of all, the corpus they use to build the translation sys- tem has a very small vocabulary. Secondly, although their algorithm is able to handle word substitution ciphers with limited vocabulary, its deciphering ac- curacy is low. The contributions of this work are: • We improve previous decipherment work by in- troducing a more efficient sampling algorithm. In experiments, our new method improves de- ciphering accuracy from 82.5% to 88.1% on (Ravi and Knight, 2011b)’s domain specific data set. Furthermore, we also solve a very large word substitution cipher built from the English Gigaword corpus and achieve 92.2% deciphering accuracy on news text. • With the ability to handle a much larger vocab- ulary, we learn a domain specific translation ta- ble from a large amount of monolingual data and use the translation table to improve out-of- domain machine translation. In experiments, we observe significant gains of up to 3.8 BLEU points. Unlike previous works, the translation table …

Content truncated to 500 words