Improved Statistical Machine Translation Using Monolingually-Derived Paraphrases

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 381–390, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Improved Statistical Machine Translation Using Monolingually-Derived Paraphrases Yuval Marton,∗ Chris Callison-Burch,† and Philip Resnik∗ ∗Department of Linguistics and the CLIP Lab at the Institute for Advanced Computer Studies (UMIACS) University of Maryland College Park, MD 20742-7505, USA {ymarton,resnik}@umiacs.umd.edu †Computer Science Department, Johns Hopkins University 3400 N. Charles Street (CSEB 226-B) Baltimore, MD 21218 ccb@cs.jhu.edu Abstract Untranslated words still constitute a ma- jor problem for Statistical Machine Trans- lation (SMT), and current SMT systems are limited by the quantity of parallel training texts. Augmenting the training data with paraphrases generated by pivot- ing through other languages alleviates this problem, especially for the so-called “low density” languages. But pivoting requires additional parallel texts. We address this problem by deriving paraphrases monolin- gually, using distributional semantic simi- larity measures, thus providing access to larger training resources, such as compa- rable and unrelated monolingual corpora. We present what is to our knowledge the first successful integration of a colloca- tional approach to untranslated words with an end-to-end, state of the art SMT sys- tem demonstrating significant translation improvements in a low-resource setting. 1 Introduction Phrase-based systems, flat and hierarchical alike (Koehn et al., 2003; Koehn, 2004b; Koehn et al., 2007; Chiang, 2005; Chiang, 2007), have achieved a much better translation coverage than word- based ones (Brown et al., 1993), but untranslated words remain a major problem in SMT. For ex- ample, according to Callison-Burch et al. (2006), a SMT system with a training corpus of 10,000 words learned only 10% of the vocabulary; the same system learned about 30% with a training corpus of 100,000 words; and even with a large training corpus of nearly 10,000,000 words it only reached about 90% coverage of the source vocab- ulary. Coverage of higher order n-gram levels is even harder. This problem plays a major part in re- ducing machine translation quality, as reflected by both automatic measures such as BLEU (Papineni et al., 2002) and human judgment tests. Improving translation coverage accurately is therefore impor- tant for SMT systems. The first solution that might come to mind is to use larger parallel training corpora. However, current state-of-the-art SMT systems cannot learn from non-aligned corpora, while sentence-aligned parallel corpora (bitexts) are a limited resource (See Section 2 for discussion of automatically- compiled bitexts). Another direction might be to make use of non-parallel corpora for training. However, this requires developing techniques to extract alignments or translations from them, and in a sufficiently fast, memory-efficient, and scal- able manner. One approach that can, in princi- ple, better exploit both alignments from bitexts and make use of non-parallel corpora is the dis- tributional collocational approach, e.g., as used by Fung and Yee (1998) and Rapp (1999). However, the systems described there are not easily scalable, and require pre-computation of a very large col- location counts matrix. Related attempts propose generating bitexts from comparable and “quasi- comparable” bilingual texts …

Content truncated to 500 words