Example-based machine translation using structural translation examples

Overview of the IWSLT04 Evaluation Campaign Yasuhiro Akiba �� , Marcello Federico �� , Noriko Kando �� , Hiromi Nakaiwa �� , Michael Paul �� , Jun’ichi Tsujii �� �� ATR, �� ITC-irst, �� NII, �� University of Tokyo yasuhiro.akiba@atr.jp, federico@itc.it, kando@nii.ac.jp hiromi.nakaiwa@atr.jp, michael.paul@atr.jp, tsujii@is.s.u-tokyo.ac.jp Abstract This paper gives an overview of the evaluation campaign results of the IWSLT041 workshop, which is organized by the C-STAR2 consortium to investigate novel speech transla- tion technologies and their evaluation. The objectives of this workshop is to provide a framework for the applicability vali- dation of existing machine translation evaluation methodolo- gies to evaluate speech translation technologies. The work- shop also strives to find new directions in how to improve current methods. 1. Introduction The drastic increase in demands for the capability to as- sist trans-lingual conversations, triggered by IT technologies such as the Internet and the expansion of borderless com- munities such as the increased number of EU countries, has accelerated research activities on speech-to-speech transla- tion technology. Many research projects have been designed to advance this technology, such as VERBMOBIL, C-STAR, NESPOLE!, and BABYLON. These projects, except for C- STAR, have mainly focused on the construction of a pro- totype system for several language pairs. On the contrary, one of C-STAR’s ongoing projects is the joint development of a speech corpus that can handle a common task in mul- tiple languages. As a first result of this activity, a Japanese- English speech corpus comprising tourism-related sentences, originally compiled by ATR, has been translated into the na- tive languages of the C-STAR members. The corpus serves as a primary source for developing and evaluating broad- coverage speech translation technologies [1]. This corpus is used in the research and development of multi-lingual speech-to-speech translation systems on a “common use” ba- sis. For the effective and efficient research and development of speech-to-speech translation systems, the evaluation of current translation quality is very important. In particular, the system developments done by using a common corpus, like C-STAR project, require careful evaluation of the prominent 1International Workshop on Spoken Language Translation, http://www.slt.atr.jp/IWSLT2004 2Consortium for Speech Translation Advanced Research, http://www.c- star.org/ translation techniques. Therefore, there is strong demand for the establishment of evaluation metrics for multilingual speech-to-speech translation systems. For this purpose, the Evaluation Campaign 2004 was carried out using parts of the multilingual corpus (cf. Sec- tion 2.1). The task was to translate 500 Chinese or Japanese sentences into English. Depending on the amount of per- mitted training data, three different language resource condi- tions (Small Data Track, Additional Data Track, Unrestricted Data Track) were distinguished. The translation quality was measured using both human assessments (subjective evalu- ation) and automatic scoring techniques (automatic evalua- tion). The evaluation results of the submitted MT systems are summarized in Section 3. The corpus supplied for this year’s conference, the refer- ence translations, the output of the participating MT systems, and the evaluation results will be made publicly available af- ter the workshop. These resources can be used as a …

Content truncated to 500 words