Study on evaluation of WWW MT systems

MT Summit VII Sept. 1999 Study on Evaluation of WWW MT Systems Shinichiro MIYAZAWA (Shumei Univ.), Shoichi YOKOYAMA (Yamagata Univ.), Masaki MATSUDAIRA (Oki), Akira KUMANO (Toshiba), Shuji KODAMA (Fujitsu), Hideki KASHIOKA (ATR), Yoshiko SHIROKIZAWA (JST), and Yasuo NAKAJIMA (AAMT) Faculty of International Cooperation, Shumei University 1-1. Daigakucyo, Yachiyo, Chiba 276, Japan Tel. +81-47-488-2111. FAX +81-47-488-8290. e-mail: miyazawa@cc.shumei-u.ac.jp Abstract Compared with off-line machine translation (MT). MT for the WWW has more evaluation factors such as translation accuracy of text, interpretation of HTML tags, consistency with various protocols and browsers, and translation speed for net surfing. Moreover, the speed of technical innovation and its practical application is fast, including the appearance of new protocols. Improvement of MT software for the WWW will enable the sharing of information from around the world and make a great deal of contribution to mankind. Despite the importance of general evaluation studies on MT software for the WWW. it appears that such studies have not yet been conducted. Since MT for the WWW will be a critical factor for future international communication, its study and evaluation is an important theme. This study aims at standardized evaluation of MT for the WWW. and suggests an evaluation method focusing on unique aspects of the WWW independent of text. This evaluation method has a wide range of aptitude without depending on specific languages. Twenty-four items specific to the WWW were actually evaluated with regard to six MT software for the WWW. This study clarified various issues which should be improved in the future regarding MT software for the WWW and issues on evaluation technology of MT on the Internet. 1. Introduction A number of machine translation (MT) software is currently on the Japan market (Table 1). and most are for translation of the WWW (World Wide Web). This basically means that the demand for reading English homepages (HP) in Japanese is very large. The most critical issue for off-line MT is how to accurately translate text. On the other hand, WWW MT has various evaluation factors such as accuracy of text translation, consistency among various protocols and browsers, translation speed, etc. Another characteristic is that the speed of technical innovation and its practical application is rapid including the appearance of new protocols. Perfection of WWW MT software will enable the sharing of information from around the world and make a great deal of contribution to mankind. Furthermore, if MT systems in other languages are adopted in the framework of WWW MT it will be possible to translate HP into various languages which will promote mutual understanding among countries. 2 Characteristics of WWW MT WWW MT, different from MT for regular text, should be able to perform the following: (1) distinguish HTML tags from text (2) understand meaning of HTML tags and translate accordingly (3) cope with various protocols including JAVA JAVA Script, and XML (4) cope with new WWW technologies which subsequently appear (5) translate in a timely manner since net surfing is performed in an accelerated manner (cost control …

Content truncated to 500 words