Parallel FDA5 for Fast Deployment of Accurate Statistical Machine Translation Systems

Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 59–65, Baltimore, Maryland USA, June 26–27, 2014. c⃝2014 Association for Computational Linguistics Parallel FDA5 for Fast Deployment of Accurate Statistical Machine Translation Systems Ergun Bic¸ici Centre for Next Generation Localisation School of Computing Dublin City University ergun.bicici@computing.dcu.ie Qun Liu Centre for Next Generation Localisation School of Computing Dublin City University qliu@computing.dcu.ie Andy Way Centre for Next Generation Localisation School of Computing Dublin City University away@computing.dcu.ie Abstract We use parallel FDA5, an efficiently pa- rameterized and optimized parallel im- plementation of feature decay algorithms for fast deployment of accurate statistical machine translation systems, taking only about half a day for each translation di- rection. We build Parallel FDA5 Moses SMT systems for all language pairs in the WMT14 translation task and obtain SMT performance close to the top Moses systems with an average of 3.49 BLEU points difference using significantly less resources for training and development. 1 Introduction Parallel FDA5 is developed for fast deployment of accurate statistical machine translation systems using an efficiently parameterized and optimized parallel implementation of feature decay algo- rithms (Bic¸ici and Yuret, 2014). Parallel FDA5 takes about half a day for each translation direc- tion. We achieve SMT performance that is on par with the top constrained Moses SMT systems. Statistical machine translation (SMT) is a data intensive problem. If you have the translations for the source sentences you are translating in your training set or even portions of it, then the trans- lation task becomes easier. If some tokens are not found in the training data then you cannot trans- late them and if some translated word do not ap- pear in your language model (LM) corpus, then it becomes harder for the SMT engine to find its cor- rect position in the translation. The importance of parallel FDA5 increases with the proliferation of training material available for building SMT sys- tems. Table 2 presents the statistics of the avail- able training and LM corpora for the constrained (C) systems as well as the statistics of the Parallel FDA5 selected training and LM corpora. Parallel FDA5 runs separate FDA5 models on randomized subsets of the training data and com- bines the selections afterwards. We run parallel FDA5 SMT experiments using Moses (Koehn et al., 2007) in all language pairs in WMT14 (Bojar et al., 2014) and obtain SMT performance close to the top constrained Moses systems training using all of the training material. Parallel FDA5 allows rapid prototyping of SMT systems for a given tar- get domain or task and can be very useful for MT in target domains with limited resources or in dis- aster and crisis situations (Lewis et al., 2011). 2 Parallel FDA5 for Instance Selection 2.1 FDA5 FDA is developed mainly for building high per- formance SMT systems using fewer yet relevant data that is selected for increasing the coverage of the test set features while maximizing their diver- sity (Bic¸ici and Yuret, 2011; Bic¸ici, 2011). Par- allel FDA parallelize instance …

Content truncated to 500 words