A Recurrent Neural Networks Approach for Estimating the Quality of Machine Translation Output

Proceedings of NAACL-HLT 2016, pages 494–498, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics A Recurrent Neural Networks Approach for Estimating the Quality of Machine Translation Output Hyun Kim Creative IT Engineering, Pohang University of Science and Technology (POSTECH), Pohang, Republic of Korea hkim.postech@gmail.com Jong-Hyeok Lee Computer Science and Engineering, Pohang University of Science and Technology (POSTECH), Pohang, Republic of Korea jhlee@postech.ac.kr Abstract This paper presents a novel approach using recurrent neural networks for estimating the quality of machine translation output. A se- quence of vectors made by the prediction method is used as the input of the final recur- rent neural network. The prediction method uses bi-directional recurrent neural network architecture both on source and target sen- tence to fully utilize the bi-directional quality information from source and target sentence. Our experiments show that the proposed re- current neural networks approach achieves a performance comparable to the existing state- of-the-art models for estimating the sentence- level quality of English-to-Spanish transla- tion. 1 Introduction Estimating the quality of machine translation output, called quality estimation (QE) (Specia et al., 2009; Blatz et al., 2004), is to pre- dict quality scores/categories for unseen machine- translated sentences without reference translations at various granularity levels (sentence-level/word- level/document-level). Quality estimation is of growing importance in the field of machine transla- tion (MT) since MT systems are widely used and the quality of each machine-translated sentence is able to vary considerably. Previous research on QE, addressed as a re- gression/classification problem to compute quality scores/categories, has mainly focused on feature ex- traction and feature selection. Feature extraction is to find the relevant features, such as baseline fea- tures (Specia et al., 2013) and latent semantic index- ing (LSI) based features (Langlois, 2015), captur- ing various aspects of quality from source and target sentences1 and external resources. Feature selection is to select the best features by using selection al- gorithms, such as Gaussian processes (Shah et al., 2015) and heuristic (Gonz´alez-Rubio et al., 2013), among already extracted features. Finding desirable features has played a key role in the QE research. In this paper we present a recurrent neural net- works approach for estimating the quality of ma- chine translation output at sentence level, which does not require manual effort for finding the best relevant features. The remainder of this paper is or- ganized as follows. In Section 2, we propose a re- current neural networks approach using a sequence of vectors made by the prediction method as input for quality estimation. And we describe the pre- diction method using bi-directional recurrent neural networks architecture in Section 3. In Section 4, we report evaluation results, and conclude our paper in Section 5. 2 Recurrent Neural Networks Approach for Estimating Quality Score Because recurrent neural networks (RNNs) have the strength for handling sequential data (Goodfellow et al., 2015), we apply RNNs to estimate the quality score of translation. The input of the final RNN is a sequence of vec- tors that have quality information about whether …

Content truncated to 500 words