Application of word-level confidence measures in interactive statistical machine translation

Application of Word-Level Confidence Measures in Interactive Statistical Machine Translation Nicola Ueffing and Hermann Ney Lehrstuhl f¨ur Informatik VI, Computer Science Department RWTH Aachen University D-52056 Aachen, Germany {ueffing,ney}@informatik.rwth-aachen.de Abstract. In this paper, we will address the question of how to efficiently integrate word confidence measures into a state-of-the-art interactive statistical machine translation system and improve prediction performance. Different methods will be presented: the selection of words according to their confidence as well as the rejection which has not been investigated so far. Experimental evaluation with respect to prediction accuracy of the system and typing effort saved by a user will show the improved prediction quality. Additionally, we will describe novel methods exploiting knowledge about a correct prefix of the target sentence for confidence estimation. These further increase the interactive system’s performance. 1 Introduction The work presented in this paper deals with the ap- plication of confidence estimation in an interactive machine translation system. This system aims at improving the productivity of human translators by suggesting translations of the source text and taking text into account that the user has typed already. We integrated confidence estimation into such an interactive system in order to improve the quality of translations predicted by the system. Since the goal is to reduce users’ effort, one has to consider the gain in keystrokes needed to type the translation as well as the time that he or she spends on reading and deciding whether to accept a suggestion. That is, the system has to keep the balance between the benefit of long predictions and the negative effect of incorrect predictions. We apply confidence esti- mation as a way to achieve this balance. The system uses confidence estimation in two dif- ferent ways: for the rejection of words with low confidence and for the selection of words for ex- tension based on their confidence. Those methods improve the prediction accuracy of the system and reduce the typing effort of a human user. We further improve these methods by taking a correct prefix that the user has already entered into account. The confidence estimation can be modified to account only for untranslated words in the source sentence. 2 Interactive Statistical Machine Transla- tion In statistical machine translation (SMT), the trans- lation is modeled as a decision process: For a given source string f J target string eI1 = f1 . . . fj . . . fJ, we seek for the 1 = e1 . . . ei . . . eI with maximal pos- terior probability: ˆe 1ˆI = argmax I,eI 1 � Pr(eI 1 | fJ 1 ) � In the setting that we investigate in this paper, a state-of-the-art SMT system is employed in an in- teractive translation system in the following way: For a given source sentence, the system generates a translation. A human translator checks this transla- tion from left to right, correcting the first error. The SMT system then proposes a new extension, taking the correct prefix ei 1 = e1 …

Content truncated to 500 words