The ISL statistical translation system for spoken language translation

The ISL Statistical Translation System for Spoken Language Translation Stephan Vogel+, Sanjika Hewavitharana+, Muntsin Kolss∗, Alex Waibel+∗, Interactive Systems Laboratories +Carnegie Mellon University, Pittsburgh, USA ∗University of Karlsruhe, Karlsruhe, Germany [vogel+, sanjika, ahw]@cs.cmu.edu, muntsin@ira.uka.de Abstract In this paper we describe the components of our statistical machine translation system used for the spoken language translation evaluation campaign. This system is based on phrase-to-phrase translations extracted from a bilingual corpus. A new phrase alignment approaches will be in- troduced, which finds the target phrase by optimizing the overall word-to-word alignment for the sentence pair un- der the constraint that words within the source phrase are only aligned to words within the target phrase. The sys- tem will be used for Chinese-to-English translations un- der the small, additional and unlimited data conditions, and for the small Japanese-to-English translation track. 1. Introduction Statistical machine translation (SMT) is currently the most promising approach, esp. to large vocabulary text translation. In the spirit of the Candide system devel- oped in the early 90s at IBM [Brown et al. 1993], a num- ber of statistical machine translation systems have been presented in the last few years [Wang and Waibel 1998], [Och and Ney 2000], [Yamada and Knight 2000]. These systems share the basic underlying principles of apply- ing a translation model to capture the lexical and word reordering relationships between two languages, comple- mented by a target language model to drive the search process through translation model hypotheses. Their pri- mary differences lie in the structure and source of their translation models. Whereas the original IBM system was based on purely word-based translation models, cur- rent SMT systems try to incorporate more complex struc- ture. The statistical machine translation system devel- oped in the Interactive Systems Laboratories (ISL) uses phrase-to-phrase translations as the primary building blocks to capture local context information, leading to better lexical choice and more reliable local reordering. A new approach to extract phrase translation pairs from bilingual data has been developed, which is not using the Viterbi alignment, but is based on optimizing a constraint word-to-word alignment for the entire sentence pair. This is described in Section 2.1. Finding good phrase translation pairs is very impor- tant. But as a source phrase can have alternative trans- lations, it is also necessary to assign meaningful prob- abilities to those alternatives. Typically, longer phrases are seen only a few times. Probabilities estimated from relative frequencies are therefore not reliable. We there- fore calculate phrase translation probabilities based on the word-to-word translation probabilities, as described in Section 2.3. Section 3 outlines the architecture of the decoder that combines the translation and language model to generate complete translations. The BTEC corpus is a very limited domain corpus and therefore many test sentences are close to one or several sentences seen in the training data. We imple- mented and tested a simple translation memory compo- nent, which will be described in Section 4. Finally, in Section 5 we present a series of ex- periments in the Chinese-to-English and …

Content truncated to 500 words