An empirical study of segment prioritization for incrementally retrained post-editing-based SMT

An Empirical Study of Segment Prioritization for Incrementally Retrained Post-Editing-Based SMT Jinhua Du jdu@computing.dcu.ie Ankit Srivastava asrivastava@computing.dcu.ie Andy Way away@computing.dcu.ie ADAPT Centre, School of Computing, Dublin City University, Ireland Alfredo Maldonado-Guerra maldonaa@tcd.ie David Lewis dave.lewis@cs.tcd.ie ADAPT Centre, Trinity College Dublin, Ireland Abstract Post-editing the output of a statistical machine translation (SMT) system to obtain high-quality translation has become an increasingly common application of SMT, which henceforth we re- fer to as post-editing-based SMT (PE-SMT). PE-SMT is often deployed as an incrementally retrained system that can learn knowledge from human post-editing outputs as early as possible to augment the SMT models to reduce PE time. In this scenario, the order of input segments plays a very important role in reducing the overall PE time. Under the active learning-based (AL) framework, this paper provides an empirical study of several typical segment prioriti- zation methods, namely the cross entropy difference (CED), n-grams, perplexity (PPL) and translation confidence, and verifies their performance on different data sets and language pairs. Experiments in a simulated setting show that the confidence of translations performs best with decreases of 1.72-4.55 points TER absolute on average compared to the sequential PE-based incrementally retrained SMT. 1 Introduction In recent years, SMT systems have been widely deployed into the translator’s workflow in the localization and translation industry to improve productivity, refereed to as post-editing-based SMT. However, in most cases, current SMT systems cannot generate high-quality translations, so human effort is usually required. With the help of incrementally improved SMT systems, the productivity of translators/post-editors can be significantly increased due to the early learning of knowledge from the previously post-edited segments (Guerberof, 2009; Plitt and Masselot, 2010; Carl et al., 2011; OBrien, 2011; Zhechev, 2012; Guerberof, 2013). Furthermore, the order of input segments has been found to have a significant impact on the overall PE-time, i.e., an optimized sequence of input segments can reduce the overall PE-time compared to the typical chronological sequence (Dara et al., 2014). Regarding the PE-SMT, the incremental retraining can be roughly categorized into two dif- ferent scenarios, namely the segment-level online incremental retraining (segment mode) (Lev- enberg et al., 2010; Denkowski et al., 2014) and batch-level incremental retraining (batch mod- e) (Hardt and Elming, 2010; Henr´ıquez Q. et al., 2011; Mathur et al., 2013; Simard and Foster, 2013; Dara et al., 2014; Bertoldi et al., 2014). The former takes one post-edited segment per re- training cycle to immediately update the models, which requires rapid incremental processing of Proceedings of MT Summit XV, vol.1: MT Researchers' Track Miami, Oct 30 - Nov 3, 2015 | p. 172 the word alignment, phrase/rule generation, language model and parameters tuning etc., while the latter firstly accumulates a batch of segments, and then performs the incremental retraining process to update the system. The batch-level mode can perform the incremental retraining process in the background while the translators/post-editors continue to work on the next batch of segments. From the point of view of parameter estimation, the former can promptly adapt its feature weights …

Content truncated to 500 words