IBM spoken language translation system evaluation

IBM Spoken Language Translation System Evaluation Young-Suk Lee, Salim Roukos IBM T. J. Watson Research Center Yorktown Heights, NY 10598 {ysuklee, roukos}@us.ibm.com Abstract We discuss phrase-based statistical machine translation performance enhancing techniques which have proven effective for Japanese-to-English and Chinese-to- English translation of BTEC corpus. We also address some issues that arise in conversational speech translation quality evaluations. 1. Introduction IBM spoken language translation system is based on a statistical translation model introduced in [1]. We adopt a phrase translation model as the baseline, for which the unit of translation is a phrase consisting of one or more words, [2], [3], [4], [5], [6]. The baseline system is augmented by the morphological analysis detailed in [7] for an improved word alignment and phrase selection. System performance is significantly improved by phrase selection from recall oriented word alignments (see Section 2 for the definition) and filtering. Re-ordering of source language sentence into the target language word order, [21], [22], further improves phrase selection and word order accuracy. Non-monotone decoding and language model probability computation for every word in a target phrase enhances the translation quality over monotone decoding and language model probability computation only for words at phrase boundaries. In Section 2, we give an overview of the baseline system. In Section 3, we discuss translation quality enhancing techniques along with experimental results. In Section 4, we address some issues in conversational speech translation evaluation. We discuss future work in Section 5. We use the term block (b) to denote a phrase translation pair consisting of a source phrase ( f ) and a target phrase ( e ). We use the symbol Pr(·) to denote general probability distribution and p(·) to denote model-based probability distribution. 2. Baseline System Overview Our baseline phrase translation system described in [Tillmann 2003] consists of three major components: word alignment, block selection, and decoding. 2.1. Word Alignment We obtain word alignment between the source and the target language sentences by successive application of IBM Model 1 viterbi alignment for initialization and iterative HMM-based alignment, [8], for refinement. We align a parallel corpus bi-directionally: one from the source language to the target language (A1: f → e) and the other from the target language to the source language (A2: e → f), where f denotes a source word position and e a target word position. We define precision (AP) and recall (AR) oriented alignments as follows: AP = A1 ∩ A2 AR = A1 U A2 AP is the intersection of A1 and A2, a high precision alignment. AR is the union of A1 and A2, a high recall alignment. The set of all source word positions covered by some word links in AP are denoted as col(AP). 2.2. Block Selection Starting from a high precision word alignment AP, we obtain blocks according to (i) a projection algorithm and (ii) a block extension algorithm. Projection Algorithm: We first project source intervals [f´, f], where f´, f ∈ col(AP). We compute the minimum target index e´ and …

Content truncated to 500 words