Modular MT with a Learned Bilingual Dictionary: Rapid Deployment of a New Language Pair

Modular MT with a learned bilingual dictionary: rapid deployment of a new language pair Jessie Pinkham Martine Smets Microsoft Research One Microsoft Way Redmond, WA 98052 {jessiep martines}@microsoft.com Abstract The MT system described in this paper combines hand-built analysis and generation components with automatically learned example-based transfer patterns. Up to now, the transfer component used a traditional bilingual dictionary to seed the transfer pattern learning process and to provide fallback translations at runtime. This paper describes an improvement to the system by which the bilingual dictionary used for these purposes is instead learned automatically from aligned bilingual corpora, making the system’s transfer knowledge entirely derivable from corpora. We show that this system with a fully automated transfer process performs better than the system with a hand-crafted bilingual dictionary. More importantly, this has enabled us to create in less than one day a new language pair, French-Spanish, which, for a technical domain, surpasses the quality bar of the commercial system chosen for comparison. 1 Introduction The phrase “MT in a day” is strongly associated with research in statistical MT. In this paper we demonstrate that “MT in a day” is possible with a non-statistical MT system provided that the transfer component is learned from aligned bilingual corpora (bi-texts), and does not rely on any large hand-crafted bilingual resource. We propose instead to use a bilingual dictionary learned only from the same bi-texts. Section 4.2 describes the creation of the new language pair, French-Spanish, and gives evaluation results. Section 4.1 examines the impact of the learned dictionary on our existing French-English system. 2 Previous work Commercial systems and other large-scale systems have traditionally relied heavily on the knowledge encoded in their bilingual dictionaries. Gerber & Yang (1997) clearly state that Systran’s translation capabilities are dependent on “large, carefully encoded, high- quality dictionaries”. With the advent of bi- texts, efforts to derive bilingual lexicons have led to substantial research (Melamed 1996, Moore 2001 for discussion), including resources for semi-automatic creation of bilingual lexica such as SABLE (Melamed 1997), used for instance in Palmer et al. (1998). Statistical MT systems have relied on bi-texts to automatically create word-alignments; in many statistical MT systems however, the authors state that use of a conventional bilingual dictionary enhances the performance of the system (Al-Onaizan et al. 1999, Koehn & Knight 2001). We find then, that in spite of the movement to create bilingual dictionaries automatically, there is still a heavy reliance on hand-crafted and hand-edited resources. We found no full-scale MT system that relied only on learned bilingual dictionaries and certainly none that was found better in performance for doing so. Rapid deployment of a new language pair has been one of the strong features of statistical MT systems. For example, “MT in a day” was a stated goal of the workshop on statistical MT (Al-Onaizan et al. 1999). The system deployed was of low quality, in part because of the small size of the corpus used, and the difficulty of the language pair chosen (Chinese to …

Content truncated to 500 words