Data selection for discriminative training in statistical machine translation

Data Selection for Discriminative Training in Statistical Machine Translation Xingyi Song and Lucia Specia Department of Computer Science University of Sheffield S1 4DP, UK {xsong2,l.specia}@sheffield.ac.uk Trevor Cohn Computing and Information Systems The University of Melbourne VIC 3010, Australia t.cohn@unimelb.edu.au Abstract The efficacy of discriminative training in Statistical Machine Translation is heavily dependent on the quality of the develop- ment corpus used, and on its similarity to the test set. This paper introduces a novel development corpus selection algo- rithm – the LA selection algorithm. It fo- cuses on the selection of development cor- pora to achieve better translation quality on unseen test data and to make training more stable across different runs, particu- larly when hand-crafted development sets are not available, and for selection from noisy and potentially non-parallel, large scale web crawled data. LA does not re- quire knowledge of the test set, nor the de- coding of the candidate pool before the se- lection. In our experiments, development corpora selected by LA lead to improve- ments of over 2.5 BLEU points when com- pared to random development data selec- tion from the same larger datasets. 1 Introduction Discriminative training – also referred to as tuning – is an important step in log-linear model in Sta- tistical Machine Translation (SMT) (Och and Ney, 2002). The efficacy of training is closely related to the quality of training samples in the develop- ment corpus, and to a certain extent, to the prox- imity between this corpus and the test set(s). Hui et al. (2010) in their experiments show that by us- ing different development corpora to train the same c⃝ 2014 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. SMT system, translation performance can vary up to 2.5 BLEU points (Papineni et al., 2002) with a standard phrase-based system (Koehn et al., 2007). How to build a ‘suitable’ development corpus is a important problem in SMT discriminative training. A suitable development corpus should aid dis- criminative training achieve higher quality mod- els, and thus yield better translations. Previous re- search on selecting training samples for the devel- opment corpus can be grouped into two categories: i) selecting samples based on the test set (trans- ductive learning), or ii) selecting samples without knowing the test set (inductive learning). Research in the first category focuses on how to find simi- lar samples to the ones the system will be tested on. Li et al. (2010), Lu et al. (2008), Zheng et al. (2010), and Tamchyna et al. (2012) measure similarity based on information retrieval methods, while Zhao et al. (2011) selects similar sentences based on edit distance. These similarity based ap- proaches have been successfully applied to the lo- cal discriminative algorithm proposed in (Liu et al., 2012). The limitation of these approaches is that the test set needs to be known before model building, which is rarely true in practice. Our research belongs to the second category. Previous work on development data …

Content truncated to 500 words