Mining a Comparable Text Corpus for a Vietnamese-French Statistical Machine Translation System

Proceedings of the Fourth Workshop on Statistical Machine Translation , pages 165–172, Athens, Greece, 30 March – 31 March 2009. c⃝2009 Association for Computational Linguistics Mining a comparable text corpus for a Vietnamese - French statistical machine translation system Thi-Ngoc-Diep Do *,**, Viet-Bac Le *, Brigitte Bigi*, Laurent Besacier*, Eric Castelli** *LIG Laboratory, CNRS/UMR-5217, Grenoble, France ** MICA Center, CNRS/UMI-2954, Hanoi, Vietnam thi-ngoc-diep.do@imag.fr Abstract This paper presents our first attempt at con- structing a Vietnamese-French statistical machine translation system. Since Vietnam- ese is an under-resourced language, we con- centrate on building a large Vietnamese- French parallel corpus. A document align- ment method based on publication date, spe- cial words and sentence alignment result is proposed. The paper also presents an appli- cation of the obtained parallel corpus to the construction of a Vietnamese-French statis- tical machine translation system, where the use of different units for Vietnamese (sylla- bles, words, or their combinations) is dis- cussed. 1 Introduction Over the past fifty years of development, ma- chine translation (MT) has obtained good results when applied to several pairs of languages such as English, French, German, Japanese, etc. How- ever, for under-resourced languages, it still re- mains a big gap. For instance, although Vietnamese is the 14th widely-used language in the world, research on MT for Vietnamese is very rare. The earliest MT system for Vietnamese is the system from the Logos Corporation, developed as an English-Vietnamese system for translating aircraft manuals during the 1970s (Hutchins, 2001). Until now, in Vietnam, there are only four research groups working on MT for Vietnamese- English (Ho, 2005). However the results are still modest. MT research on Vietnamese-French occurs even more rarely. Doan (2001) proposed a trans- lation module for Vietnamese within ITS3, a multilingual MT system based on the classical analysis-transfer-generation approach. Nguyen (2006) worked on Vietnamese language and Vietnamese-French text alignment. But no com- plete MT system for this pair of languages has been published so far. There are many approaches for MT: rule-based (direct translation, interlingua-based, transfer- based), corpus-based (statistical, example-based) as well as hybrid approaches. We focus on build- ing a Vietnamese-French statistical machine translation (SMT) system. Such an approach re- quires a parallel bilingual corpus for source and target languages. Using this corpus, we build a statistical translation model for source/target lan- guages and a statistical language model for target language. Then the two models and a search module are used to decode the best translation (Brown et al., 1993; Koehn et al., 2003). Thus, the first task is to build a large parallel bilingual text corpus. This corpus can be de- scribed as a set of bilingual sentence pairs. At the moment, such a large parallel corpus for Viet- namese-French is unavailable. (Nguyen, 2006) presents a Vietnamese-French parallel corpus of law and economics documents. Our SMT system was trained using Vietnamese-French news cor- pus created by mining a comparable bilingual text corpus from the Web. Section 2 presents the general methodology of mining a comparable text corpus. We present …

Content truncated to 500 words