MT engine selection in the enterprise

Machine Translation engine selection in the Enterprise Heidi Depraetere Cross Language NV Woodrow Wilsonplein 7 9000 Gent, Belgium heidi.depraetere@crosslang.com Pablo Vazquez Cisco Systems, Inc. 150 West Tasman Drive San Jose, CA USA 95134 pavazque@cisco.com Abstract The focus of this article is to show how client centric Machine Translation (MT) evaluation can assist in identifying the best MT solution in an Enterprise con- text. Examining extensive lists of opaque scores (BLEU and others) does not bring value to the Enterprise market unless it can equate the results with its own daily commercial challenges. Armed with the right set of directives the decision process is made simpler as Cisco recently discovered. From the re- sults of an external evaluation program they were able to identify a clear process for engine selection as well as creating a matrix for future needs. 1 Introduction Enterprise evaluation of MT is often assigned to people and teams that have little knowledge or experience of how MT is deployed in their busi- nes environment. This paper aims at showing how client centred evaluation leads to defining minimum acceptance criteria for MT engine se- lection purposes. The findings are based on quantitative and qu- alitative data gathered during an evaluation process carried out for Cisco, by an external partner. The issue of return of investment and where MT can be used is outside the scope of this pa- per. These subjects although very important and critical for the adoption of a technology like MT should be done prior to the engine selection and may be the subject for another paper. The starting point of this article is the Enter- prise with a clearly identified MT application context including content to be translated as well as an explicit MT workflow. 2 Parameters Clearly one needs to evaluate the potential of a new technology and understand what it can do for the Enterprise before one considers using it. MT is no different; except that once one goes through this process and decide that it can bring a benefit to the Enterprise one is then faced with an additional challenge: identifying which MT sys- tem(s) on offer will best meet one’s specific needs. This is well documented by (Lehrberger & Bourbeau, 1988): 1 “The objective of an eval- uation is of course to determine whether a system permits an adequate response to given needs and constraints.” It is therefore essential to define a set of pa- rameters which will allow the main stakeholder and potential users in the Enterprise to make an objective business decision. 3 Defining Cisco’s needs Cisco has been using MT for translating support content for nearly a decade. They translate tech- nical documentation and troubleshooting guides to aid their customers and partners in solving problems without the need to directly contact Customer Support. All the material that has been tagged translata- ble is currently published in Spanish, Portuguese, Russian, Japanese and Chinese. Before the launch of each new website the content is 1/Lehrberger/ J., /Bourbeau/ L. 1988. …

Content truncated to 500 words