Deploying the SAE J2450 translation quality metric in MT projects

MT Summit VII Sept. 1999 Deploying the SAE J2450 Translation Quality Metric in MT Projects Jörg Schütz IAI Saarbrücken Martin-Luther-Str. 14 D-66111 Saarbrücken Germany email: joerg@iai.uni-sb.de Abstract This paper provides a nutshell description of how the recently published proposal of a translation quality metric for automotive service information is applicable in an evalua- tion scenario that deploys multilingual human language technology (mHLT). This proposal is the result of the J2450 task force group of the Society of Automotive Engineers (SAE). The main focus of the developed metric is on the syntactic level of a translation product. Since it is our belief that any evaluation of a translation (human and machine) should also take into account the semantic level of a hu- man language product, we have slightly re- shaped the SAE J2450 metric. In addition, we have embedded the whole evaluation process into an object-oriented quality model approach to account for the established business proc- esses in the acquisition, production, translation and dissemination of automotive service in- formation in SGML/XML environments. This scenario then provides the solid ground- ing for the setup of a quality assurance process for all dimensions related to the processing (human and machine) of automotive service information. The work reported here is one part of the ongoing European Multidoc project that has brought together several European automotive companies to taming the complex- ity of service information products in an inte- grated way. Within Multidoc integration means first and foremost the coupling of ad- vanced information technology and mHLT. These aspects will be further motivated and detailed in the context of the specification of an evaluation scenario. 1 Introduction Automotive documentation is faced with a tremendous increase of translating technical information into mul- tiple languages. Since most of the translation tasks are contracted out to translation companies, there are no efficient and effective measures for controlling the translation process and for benchmarking the transla- tion quality. This is an unfortunate situation because the quality of a company’s technical documentation is an important added value resource and one basis of competitive advantage, including the various legal im- pacts of inaccurate and inconsistent technical informa- tion. In 1997 the Society of Automotive Engineers (SAE) has initiated a task force with the aim to establish a translation quality metric that could be used by auto- motive companies to compare the quality of service information translation deliverables. This task force has recently published a proposal which defines eight quality metrics for language translation of automotive service information [SAE J2450, 1999]. Currently, the proposal covers only the syntactic aspects of a transla- tion product, and thus has to be seen as a first step to- wards a standard for defining and proofing translation quality of automotive service information. Since this activity will have a broader scope and im- pact on the quality of technical information as an ac- companying product of a technical product, it is an interesting investigation to view this work in combina- tion with the deployment of multilingual human …

Content truncated to 500 words