Regression with Phrase Indicators for Estimating MT Quality

Proceedings of the 7th Workshop on Statistical Machine Translation, pages 152–156, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics Regression with Phrase Indicators for Estimating MT Quality∗ Chunyang Wu Hai Zhao† Center for Brain-Like Computing and Machine Intelligence, Department of Computer Science and Engineering, Shanghai Jiao Tong University chunyang506@sjtu.edu.cn, zhaohai@cs.sjtu.edu.cn Abstract We in this paper describe the regression sys- tem for our participation in the quality estima- tion task of WMT12. This paper focuses on exploiting special phrases, or word sequences, to estimate translation quality. Several feature templates on this topic are put forward sub- sequently. We train a SVM regression model for predicting the scores and numerical results show the effectiveness of our phrase indicators and method in both ranking and scoring tasks. 1 Introduction The performance of machine translation (MT) sys- tems has been considerable promoted in the past two decades. However, since the quality of the sentence given by MT decoder is not guaranteed, an impor- tant issue is to automatically predict or identify its characteristics. Recent studies on quality estimation or confidence estimation have focused on measuring the translating quality at run-time, instead of involv- ing reference corpus. Researches on this topic con- tribute to offering advices or warnings for users even without knowledge about either side of languages ∗This work was partially supported by the National Natu- ral Science Foundation of China (Grant No. 60903119 and Grant No. 61170114), the National Research Foundation for the Doctoral Program of Higher Education of China under Grant No. 20110073120022, the National Basic Research Pro- gram of China (Grant No. 2009CB320901), the Science and Technology Commission of Shanghai Municipality (Grant No. 09511502400), and the European Union Seventh Framework Program (Grant No. 247619). †corresponding author and illuminating some other potential MT applica- tions. This paper describes the regression system for our participation in the WMT12 quality estimation task. In this shared task, we analyzed the pattern of trans- lating errors and studied on capturing such patterns among the corpus. The basic objective in this pa- per is to recognize those phrases, or special word se- quence combinations which can indicate the quality of a translation instance. By introducing no exter- nal NLP toolkits, we exploited several feasible tech- niques to extract such patterns directly on the cor- pus. One contribution of this paper is those feature templates on the basis of this topic. Numerical re- sults show their positive effects on both ranking and scoring subtasks. The rest of this paper is organized as follows: In Section 2, we show the related work. In Section 3, we specify the details of our system architecture. The experimental results are reported in Section 4. Finally, the conclusion is given in Section 5. 2 Related Work Compared with traditional MT metrics such as BLEU (Papineni et al., 2002), the fundamental goal of quality estimation (QE) is predicting the quality of output sentences without involving reference sen- tences. Early works (Quirk, 2004; Gamon et al., 2005) have demonstrated the consistency of the …

Content truncated to 500 words