Achieving commercial-quality translation with example-based methods

Achieving commercial-quality translation with example-based methods Stephen D. Richardson, William B. Dolan, Arul Menezes, Jessie Pinkham, Microsoft Research One Microsoft Way Redmond, WA 98052 {steveri, billdol, arulm, jessiep}@microsoft.com Abstract We describe MSR-MT, a large-scale example-based machine translation system under development for several language pairs. Trained on aligned English-Spanish technical prose, a blind evaluation shows that MSR-MT’s integration of rule-based parsers, example based processing, and statistical techniques produces translations whose quality in this domain exceeds that of uncustomized commercial MT systems. Keywords Example-based machine translation, EBMT, Hybrid MT, Example-base learning, Spanish-English MT 1. Introduction Currently available commercial machine translation (MT) systems rely on hand-coded transfer components which are both difficult and expensive to customize for a particular domain, a fact which has limited their cost- effectiveness and overall utility. To address this bottleneck, a variety of example based machine translation (EBMT) systems have been created and described in the literature (for a comprehensive overview of work in this area, see Somers 1999). This data-driven approach relies on automated or semi- automated techniques to extract translation knowledge from bilingual corpora. The field of EBMT has generated interesting experimental results, some of it based on quite large datasets (e.g. Frederking & Brown, 1996). What has so far been lacking, however, is an empirical demonstration that the quality bar set by existing commercial MT systems can be matched or surpassed by an MT system whose primary source of translation knowledge is an automatically-constructed example base. This paper reports on MSR-MT (Microsoft Research Machine Translation), a translation system that relies on EBMT (and some statistical) techniques to automatically acquire its primary translation knowledge from a bilingual corpus of several million words. MSR-MT leverages the linguistic generality of existing rule-based parsers to enable broad coverage and to overcome some of the limitations on locality of context characteristic of data- driven approaches. The quality of MSR-MT’s output for the domain to which it has been customized is shown to exceed the output quality of two highly rated (though not fully domain-customized) commercial MT systems. 2. MSR-MT MSR-MT is a data-driven hybrid MT system, combining rule-based analysis and generation components with example-based transfer. The automatic alignment procedure used to create the example base relies on the same parser employed during analysis and also makes use of its own small set of rules for determining permissible alignments. Moderately sized bilingual dictionaries, containing only word pairs and their parts of speech, provide translation candidates for the alignment procedure and are also used as a backup source of translations during transfer. Statistical techniques supply additional translation pair candidates for alignment and identify certain multi-word terms for parsing and transfer. The robust, broad-coverage parsers used by MSR-MT were created originally for monolingual applications. These parsers produce a logical form (LF) representation that is compatible across multiple languages (see section 3 below). Parsers now exist and are under active development for seven languages (English, French, German, Spanish, Chinese, Japanese, and Korean). Generation components are currently being developed for English, Spanish, Chinese, and Japanese. …

Content truncated to 500 words