A Program for Aligning Sentences in Bilingual Corpora

A PROGRAM FOR ALIGNING SENTENCES IN BILINGUAL CORPORA William A. Gale Kenneth W. Church AT&T Bell Laboratories 600 Mountain Avenue Murray Hill, NJ, 07974 ABSTRACT Researchers in both machine Iranslation (e.g., Brown et al., 1990) and bilingual lexicography (e.g., Klavans and Tzoukermann, 1990) have recently become interested in studying parallel texts, texts such as the Canadian Hansards (parliamentary proceedings) which are available in multiple languages (French and English). This paper describes a method for aligning sentences in these parallel texts, based on a simple statistical model of character lengths. The method was developed and tested on a small trilingual sample of Swiss economic reports. A much larger sample of 90 million words of Canadian Hansards has been aligned and donated to the ACL/DCI. 1. Introduction Researchers in both machine lranslation (e.g., Brown et al, 1990) and bilingual lexicography (e.g., Klavans and Tzoukermann, 1990) have recently become interested in studying bilingual corpora, bodies of text such as the Canadian I-lansards (parliamentary debates) which are available in multiple languages (such as French and English). The sentence alignment task is to identify correspondences between sentences in one language and sentences in the other language. This task is a first step toward the more ambitious task finding correspondances among words. I The input is a pair of texts such as Table 1. 1. In statistics, string matching problems are divided into two classes: alignment problems and correspondance problems. Crossing dependencies are possible in the latter, but not in the former. Table 1: Input to Alignment Program English According to our survey, 1988 sales of mineral water and soft drinks were much higher than in 1987, reflecting the growing poptdm'ity of these products. Cola drink manufacturers in particular achieved above-average growth rates. The higher turnover was largely due to an increase in the sales volume. Employment and investment levels also climbed. Following a two-year Iransitional period, the new Foodstuffs Ordinance for Mineral Water came into effect on April 1, 1988. Specifically, it contains more stringent requirements regarding quality consistency and purity guarantees. French Quant aux eaux rain&ales et aux limonades, elles rencontrent toujours plus d'adeptes. En effet, notre sondage fait ressortir des ventes nettement SUl~rieures h celles de 1987, pour les boissons base de cola notamment. La progression des chiffres d'affaires r~sulte en grande partie de l'accroissement du volume des ventes. L'emploi et les investissements ont 8galement augmentS. La nouvelle ordonnance f&16rale sur les denr6es alimentaires concernant entre autres les eaux min6rales, entree en vigueur le ler avril 1988 aprbs une p6riode transitoire de deux ans, exige surtout une plus grande constance dans la qualit~ et une garantie de la puret& The output identifies the alignment between sentences. Most English sentences match exactly one French sentence, but it is possible for an English sentence to match two or more French sentences. The first two English sentences (below) illustrate a particularly hard case where two English sentences align to two French sentences. No smaller alignments are possible because the clause "... sales ... were higher..." in …

Content truncated to 500 words