Introduction to China’s HTRDP Machine Translation Evaluation

Introduction to China’s HTRDP Machine Translation Evaluation LIU Qun, HOU Hongxu, LIN Shouxun, QIAN Yueliang Instititue of Computing Technology, Chinese Academy of Sciences No.6, Kexueyuan Nanlu, Zhongguancun Beijing 100080, China liuqun@ict.ac.cn ZHANG Yujie ISAHARA Hitoshi, National Institute of Information and Communications Technology 3-5 Hikaridai, Seika-Cho, Soraku-gun, Kyoto, 619-0289 Japan yujie@nict.go.jp Abstract Since 1994, China’s HTRDP machine translation evaluation has been conducted for five times. Systems of various translation directions between Chinese, English, Japanese and French have been tested. Both human evaluation and automatic evaluation are conducted in HTRDP evaluation. In recent years, the evaluation was organized jointly with NICT of Japan. This paper introduces some details of this evaluation. 1 Introduction Evaluation is recognized as an important drive for machine translation research. DARPA have a long history to organize machine translation evaluation. Recent years, inspired by the efficient work of the DARPA-supported NIST evaluation, some other machine translation evaluations are supposed, such as ISWLT and TC-STAR MT evaluation. In China, the HTRDP machine translation evaluation also has a long history. Since 1994, HTRDP MT evaluations have been conducted for five times. We will give a detailed introduction to China’s HTRDP MT evaluation. 2 Origination and history HTRDP means China’s national High-Tech Research and Development Programme. The full name of HTRDP evaluation is “HTRDP Evaluation on Chinese Information Processing and Intelligent Human-Machine Interface Technology”1. It is a series of evaluation activities which is sponsored by HTRDP. HTRDP 1 HTRDP is also called “863” Programme, in order to commemorate the time “March 1986” when China’s previous leader Deng Xiaoping approved the suggestion of four famous Chinese scientists to found such a programme. So the HTRDP evaluation is also called “863” evaluation. Please refer to the website: http://www.863data.org.cn evaluation covers a wide range of technologies, which include: ♦ Machine translation (MT) ♦ Automatic speech recognition (ASR) ♦ Speech to text (TTS) ♦ Chinese character recognition (CR) ♦ Information retrieval (IR) ♦ Chinese word segmentation (CWS, includes part of speech tagging and named entity recognition) ♦ Text classification (TC) ♦ Text summarization (TS) ♦ Human face detection and recognition (FR) Table 1 gives the year of each HTRDP evaluation and technology categories which were tested in that year. Please note that the 2005 HTRDP evaluation is ongoing. We can see that machine translation is firstly test in the 3rd HTRDP evaluation in 1994, and five HTRDP MT evaluations has been conducted up to now. 1st 2nd 3rd 4th 5th 6th 7th 8th 1991 1992 1994 1995 1998 2003 2004 2005 ASR ● ● ● ● ● ● ● ● TTS ● ● ● ● ● MT ● ● ● ● ● ● CWS ● ● ● ● IR ● ● ● TC ● ● TS ● ● ● ● CR ● ● ● ● ● ● FR ● Table 1 Technology categories tested in each HTRDP evaluation 3 Organizer HTRDP evaluation is organized by Institute of Computing Technology (ICT), Chinese Academy of Sciences. Since 2004, ICT started its cooperation with the National Institute of …

Content truncated to 500 words