Stanford University’s Submissions to the WMT 2014 Translation Task

Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 150–156, Baltimore, Maryland USA, June 26–27, 2014. c⃝2014 Association for Computational Linguistics Stanford University’s Submissions to the WMT 2014 Translation Task Julia Neidert∗, Sebastian Schuster∗, Spence Green, Kenneth Heafield, and Christopher D. Manning Computer Science Department, Stanford University {jneid,sebschu,spenceg,heafield,manning}@cs.stanford.edu Abstract We describe Stanford’s participation in the French-English and English-German tracks of the 2014 Workshop on Statisti- cal Machine Translation (WMT). Our sys- tems used large feature sets, word classes, and an optional unconstrained language model. Among constrained systems, ours performed the best according to uncased BLEU: 36.0% for French-English and 20.9% for English-German. 1 Introduction Phrasal (Green et al., 2014b) is a phrase-based ma- chine translation system (Och and Ney, 2004) with an online, adaptive tuning algorithm (Green et al., 2013c) which allows efficient tuning of feature- rich translation models. We improved upon the basic Phrasal system with sparse features over word classes, class-based language models, and a web- scale language model. We submitted one constrained French-English (Fr-En) system, one unconstrained English-German (En-De) system with a huge language model, and one constrained English-German system without it. Each system was built using over 100,000 features and was tuned on over 10,000 sentences. This paper describes our submitted systems and discusses how the improvements affect translation quality. 2 Data Preparation & Post-Processing We used all relevant data allowed by the con- strained condition, with the exception of HindEn- Corp and Wiki Headlines, which we deemed too noisy. Specifically, our parallel data consists of the Europarl version 7 (Koehn, 2005), parallel Com- monCrawl (Smith et al., 2013), French-English UN, Giga-FrEn, and News Commentary corpora pro- vided by the evaluation. For monolingual data, we ∗These authors contributed equally. Sentences Tokens En-De 4.5M 222M Fr-En 36.3M 2.1B Table 1: Gross parallel corpus statistics after pre- processing. Constrained LM Unconstrained LM German 1.7B 38.9 B English 7.2B - Table 2: Number of tokens in pre-processed mono- lingual corpora used to estimate the language mod- els. We split the constrained English data into two models: 3.7 billion tokens from Gigaword and 3.5 billion tokens from all other sources. used the provided news crawl data from all years, English Gigaword version 5 (Parker et al., 2011), and target sides of the parallel data. This includes English from the Yandex, CzEng, and parallel Com- monCrawl corpora. For parallel CommonCrawl, we concatenated the English halves for various lan- guage pairs and then deduplicated at the sentence level. In addition, our unconstrained English-German system used German text extracted from the en- tire 2012, 2013, and winter 2013 CommonCrawl1 corpora by Buck et al. (2014). Tables 1 and 2 show the sizes of the pre- processed corpora of parallel text and monolingual text from which our systems were built. 2.1 Pre-Processing We used Stanford CoreNLP to tokenize the English and German data according to the Penn Treebank standard (Marcus et al., 1993). The French source data was tokenized similarly to the French Treebank 1http://commoncrawl.org 150 (Abeillé et al., 2003) using the Stanford …

Content truncated to 500 words