Coarse “split and lump” bilingual language models for richer source information in SMT

Coarse “split and lump” bilingual language models for richer source information in SMT Darlene Stewart Darlene.Stewart@nrc.gc.ca Roland Kuhn Roland.Kuhn@nrc.gc.ca Eric Joanis Eric.Joanis@nrc.gc.ca George Foster* George.Foster@nrc.gc.ca All authors originally at: National Research Council, Ottawa, Canada K1A 0R6 * This author is now at Google Inc., Mountain View, California 94043 Abstract Recently, there has been interest in automatically generated word classes for improving sta- tistical machine translation (SMT) quality: e.g, (Wuebker et al, 2013). We create new mod- els by replacing words with word classes in features applied during decoding; we call these “coarse models”. We find that coarse versions of the bilingual language models (biLMs) of (Niehues et al, 2011) yield larger BLEU gains than the original biLMs. BiLMs provide phrase-based systems with rich contextual information from the source sentence; because they have a large number of types, they suffer from data sparsity. Niehues et al (2011) miti- gated this problem by replacing source or target words with parts of speech (POSs). We vary their approach in two ways: by clustering words on the source or target side over a range of granularities (word clustering), and by clustering the bilingual units that make up biLMs (bitoken clustering). We find that loglinear combinations of the resulting coarse biLMs with each other and with coarse LMs (LMs based on word classes) yield even higher scores than single coarse models. When we add an appealing “generic” coarse configuration chosen on English > French devtest data to four language pairs (keeping the structure fixed, but providing language-pair-specific models for each pair), BLEU gains on blind test data against strong baselines averaged over 5 runs are +0.80 for English > French, +0.35 for French > English, +1.0 for Arabic > English, and +0.6 for Chinese > English. 1. Introduction This work aims to provide rich contextual information to phrase-based SMT, in order to miti- gate data sparsity. We cluster the basic units of the bilingual language model (biLM) of Niehues et al (2011) and of standard language models (LMs). A “generic”, symmetric config- uration chosen on English > French devtest yields BLEU gains over strong baselines on blind test data of +0.80 for English > French (henceforth “Eng>Fre”), +0.35 for French > English (“Fre>Eng”), +1.0 for Arabic > English (“Ara>Eng”), and +0.6 for Chinese > English (“Chi>Eng”). If we apply the configuration with the highest devtest score on a given language pair to blind data, the gains are +0.85 for Eng>Fre, +0.46 for Fre>Eng, and +1.2 for Ara>Eng, but still +0.6 for Chi>Eng. 1.1. Coarse bilingual language models (biLMs) for source context Though coarse biLMs are the focus of this paper, we explored other coarse models: models where words are replaced by word classes. E.g., we obtained good gains in earlier experi- ments with coarse LMs. Since others have explored that terrain before us (see 1.2), this paper focuses on the “bilingual language model” (biLM) of Niehues et al (2011). In phrase-based Al-Onaizan & Simard (Eds.) Proceedings of AMTA 2014, vol. 1: MT Researchers Vancouver, BC …

Content truncated to 500 words