Predictor-Estimator using Multilevel Task Learning with Stack Propagation for Neural Quality Estimation

Proceedings of the Conference on Machine Translation (WMT), Volume 2: Shared Task Papers, pages 562–568 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics Predictor-Estimator using Multilevel Task Learning with Stack Propagation for Neural Quality Estimation Hyun Kim† and Jong-Hyeok Lee‡ †Creative IT Engineering, ‡Computer Science and Engineering, Pohang University of Science and Technology (POSTECH), Republic of Korea †hkim.postech@gmail.com ‡jhlee@postech.ac.kr Seung-Hoon Na Computer Science and Engineering, Chonbuk National University, Republic of Korea nash@jbnu.ac.kr Abstract In this paper, we present a two-stage neu- ral quality estimation model that uses mul- tilevel task learning for translation quality estimation (QE) at the sentence, word, and phrase levels. Our approach is based on an end-to-end stacked neural model named Predictor-Estimator, which has two stages consisting of a neural word prediction model and neural QE model. To efficiently train the two-stage model, a stack prop- agation method is applied, thereby en- abling us to jointly learn the word pre- diction model and QE model in a sin- gle learning mode. In addition, we de- ploy multilevel task learning with stack propagation, where the training examples available for all QE subtasks (i.e., sen- tence/word/phrase levels) are used to train a Predictor-Estimator for a specific sub- task. All of our submissions to the QE task of WMT17 are ensembles that com- bine a set of neural models trained under different settings of varying dimensionali- ties and shuffling training examples, even- tually achieving the best performances for all subtasks at the sentence, word, and phrase levels. 1 Introduction In this paper, we describe the two-stage end-to- end neural models submitted to the Shared Task on Sentence/Word/Phrase-Level Quality Estima- tion (QE task) at the 2017 Conference on Ma- chine Translation (WMT17). The task aims at es- timating quality scores/categories for an unseen translation without a reference translation at var- ious granularities (i.e., sentence/word/phrase lev- els) (Specia et al., 2013). Our neural network-based models for sentence/word/phrase-level QE are based on Predictor-Estimator architecture (Kim et al., 2017; Kim and Lee, 2016), which is a two-stage end-to-end neural QE model. In this submission to WMT 2017, our Predictor-Estimator model is further advanced by extensively applying a stack propagation method (Zhang and Weiss, 2016) in order to efficiently train the two-stage model. The Predictor-Estimator architecture (Kim et al., 2017; Kim and Lee, 2016) is the two-stage neural QE model (Figure 1) consisting of two types of stacked neural models: 1) a neural word prediction model (i.e., word predictor) trained from additional large-scale parallel corpora and 2) a neural QE model (i.e., quality estimator) trained from quality-annotated noisy parallel corpora called QE data. The Predictor-Estimator architecture uses word prediction as a pre-task for QE. Kim et al. (2017) showed that word prediction is helpful for improving the QE per- formance. In the first stage, the word predictor, which is based on a bidirectional and bilingual recurrent neural network (RNN) language model – the modification of the attention-based RNN encoder-decoder (Bahdanau et al., 2015; Cho et al., 2014) – predicts a target word conditioned …

Content truncated to 500 words